Second harmonic generation (SHG) as an essential nonlinear optical effect, has gradually shifted its research trend toward the integration and miniaturization of photonic and optoelectronic on‐chip devices in recent years. 2D layered materials (2DLMs) open up a new research paradigm of nonlinear optics due to their large second‐order susceptibility, atomically thin structure, and perfect phase‐matching. However, 2DLMs are facing a bottleneck of weak SHG conversion efficiency limit caused by short light–matter interaction lengths at a nanoscale. Moreover, advances in integrated on‐chip SHG devices based on 2DLMs rely on the continuing development of novel strategies with tunable and efficient SHG responses. Here, this review provides a comprehensive overview of recent progress in exploring highly efficient and tunable SHG responses in 2DLMs. Various modulation and enhancement strategies for the SHG response of 2DLMs are extensively studied and systematically discussed, which can be classified into two categories: symmetry breaking and light‐matter interaction enhancement. Moreover, remaining challenges and outlooks toward further extending and realizing the practical applications of 2DLMs in nonlinear on‐chip integrated devices with SHG modulation and enhancement characteristics are discussed.