Rabinowitz Alternative for Non-cooperative Elliptic Systems on Geodesic Balls

<jats:title>Abstract</jats:title> <jats:p>The purpose of this paper is to study properties of continua (closed connected sets) of nontrivial solutions of non-cooperative elliptic systems considered on geodesic balls in <jats:inline-formula id="j_ans-2018-0012_ineq_9999"> <jats:alternatives> <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:msup> <m:mi>S</m:mi> <m:mi>n</m:mi> </m:msup> </m:math> <jats:inline-graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="graphic/j_ans-2018-0012_inl_001.png" /> <jats:tex-math>{S^{n}}</jats:tex-math> </jats:alternatives> </jats:inline-formula>. In particular, we show that if the geodesic ball is a hemisphere, then all these continua are unbounded. It is also shown that the phenomenon of global symmetry-breaking bifurcation of such solutions occurs. Since the problem is variational and <jats:inline-formula id="j_ans-2018-0012_ineq_9998"> <jats:alternatives> <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:mi>SO</m:mi> <m:mo>⁡</m:mo> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mi>n</m:mi> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:mrow> </m:math> <jats:inline-graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="graphic/j_ans-2018-0012_inl_002.png" /> <jats:tex-math>{\operatorname{SO}(n)}</jats:tex-math> </jats:alternatives> </jats:inline-formula>-symmetric, we apply the techniques of equivariant bifurcation theory to prove the main results of this article. As the topological tool, we use the degree theory for <jats:inline-formula id="j_ans-2018-0012_ineq_9997"> <jats:alternatives> <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:mi>SO</m:mi> <m:mo>⁡</m:mo> <m:mrow> <m:mo stretchy="false">(</m:mo> <m:mi>n</m:mi> <m:mo stretchy="false">)</m:mo> </m:mrow> </m:mrow> </m:math> <jats:inline-graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="graphic/j_ans-2018-0012_inl_003.png" /> <jats:tex-math>{\operatorname{SO}(n)}</jats:tex-math> </jats:alternatives> </jats:inline-formula>-invariant strongly indefinite functionals defined in [A. Gołȩbiewska and S. A. Rybicki, Global bifurcations of critical orbits of <jats:italic>G</jats:italic>-invariant strongly indefinite functionals, Nonlinear Anal. 74 2011, 5, 1823–1834].</jats:p>