As a powerful lensless imaging method, coherent diffractive imaging reconstructs an object from its diffraction pattern by computational phase retrieval algorithm. The reconstruction quality relies heavily on the coherence of the illumination and suffers from artifacts or fails under partially coherent imaging conditions, such as the illumination by broadband or multi-wavelength source. Here, we demonstrate a single-shot coherent diffractive imaging of multi-wavelength illumination by using sparse representation and modulation optimization. Sparse representation is utilized in the phase retrieval algorithm to reduce the ambiguity of solutions caused by multi-wavelength aliasing. Furthermore, phase modulation is optimized by maximum likelihood estimation based on Cramér–Rao lower bound and is applied to encode sufficient information in diffraction patterns. Simulations and experiments have been complemented to verify the effectiveness of the proposed method.