A group theoretic proof of a compactness lemma and existence of nonradial solutions for semilinear elliptic equations

Symmetry plays a basic role in variational problems (settled, e.g., in Rn or in a more general manifold), for example, to deal with the lack of compactness which naturally appears when the problem is invariant under the action of a noncompact group. In Rn, a compactness result for invariant functions with respect to a subgroup G of O (n) has been proved under the condition that the G action on Rn is compatible, see Willem (Minimax theorem. Progress in nonlinear differential equations and their applications, vol 24, Birkhäuser Boston Inc., Boston, 1996). As a first result, we generalize this and show here that the compactness is recovered for particular subgroups of the isometry group of a Riemannian manifold. We investigate also isometric action on Hadamard manifold (M, g) proving that a large class of subgroups of Iso (M, g) is compatible. As an application, we get a compactness result for “invariant” functions which allows us to prove the existence of nonradial solutions for a classical scalar equation and for a nonlocal fractional equation on Rn for n= 3 and n= 5 , improving some results known in the literature. Finally, we prove the existence of nonradial invariant functions such that a compactness result holds for some symmetric spaces of noncompact type.