We study spontaneous symmetry breaking for field algebras on Minkowski space in the presence of a condition of geometric modular action (CGMA) proposed earlier as a selection criterion for vacuum states on general space-times. We show that any internal symmetry group must commute with the representation of the Poincaré group (whose existence is assured by the CGMA) and each translationinvariant vector is also Poincaré invariant. The subspace of these vectors can be centrally decomposed into pure invariant states and the CGMA holds in the resulting sectors. As positivity of the energy is not assumed, similar results may be expected to hold for other space-times.