Successive phase transitions in a rare-earth monoaxial chiral magnet DyNi3Ga9 have been investigated by resonant x-ray diffraction. Magnetic dipole and electric quadrupole degrees of freedom arising from the large angular moment of J = 15/2, in combination with the symmetric and antisymmetric exchange interactions and the crystal field anisotropy, give rise to competing ordered phases. We show that the antiferromagnetically coupled Dy moments in the ab-plane form an incommensurate helimagnetic order with q ∼ (0, 0, 0.43) just below TN = 10 K, which further exhibits successive first-order transitions to the commensurate helimagnetic order with q = (0, 0, 0.5) at T ′ N = 9.0 K, and to the canted antiferromagnetic order with q = (0, 0, 0) at T ′′ N = 8.5 K, both with large coexistence regions. The relation of the magnetic helicity and the crystal chirality in DyNi3Ga9 is also uniquely determined. Splitting of the (6, 0, 0) Bragg peak is observed below T ′′ N , reflecting the lattice distortion due to the ferroquadrupole order. In the canted antiferromagnetic phase, a spin-flop transition takes place at 5 K when the temperature is swept in a weak magnetic field. We discuss these transitions from the viewpoint of competing energies described above.