We investigate the fatigue resistance of chemically cross-linked polyampholyte hydrogels with a hierarchical structure due to phase separation and find that the details of the structure, as characterized by SAXS, control the mechanisms of crack propagation. When gels exhibit a strong phase contrast and a low cross-linking level, the stress singularity around the crack tip is gradually eliminated with increasing fatigue cycles and this suppresses crack growth, beneficial for high fatigue resistance. On the contrary, the stress concentration persists in weakly phase-separated gels, resulting in low fatigue resistance. A material parameter, λtran, is identified, correlated to the onset of non-affine deformation of the mesophase structure in a hydrogel without crack, which governs the slow-to-fast transition in fatigue crack growth. The detailed role played by the mesoscale structure on fatigue resistance provides design principles for developing self-healing, tough, and fatigue-resistant soft materials.