Neutron diffraction measurements on a high quality single crystal of CeAu2Ge2 reveal two new magnetic structures that coexist with commensurate long range antiferromagnetic order below the Neel temperature TN 14 K. The new magnetic structures, which exhibit distinct temperature and field dependencies, are described by both commensurate and incommensurate magnetic structures and introduces new region in the H-T phase diagram. These new experimental observations provide essential information for developing a universal understanding of the magnetic properties of this class of Ce-based CeX2M 2 (X = Cu, Ag, Au; M = Ge, Si) compounds, which are prototypes for exploring quantum phase transitions and their interplay with unconventional superconductivity.