Since the first development of large‐area graphene synthesis by the chemical vapor deposition (CVD) method in 2009, CVD‐graphene has been considered to be a key material in the future electronics, energy, and display industries, which require transparent, flexible, and stretchable characteristics. Although many graphene‐based prototype applications have been demonstrated, several important issues must be addressed in order for them to be compatible with current complementary metal‐oxide‐semiconductor (CMOS)‐based manufacturing processes. In particular, metal contamination and mechanical damage, caused by the metal catalyst for graphene growth, are known to cause severe and irreversible deterioration in the performance of devices. The most effective way to solve the problems is to grow the graphene directly on the semiconductor substrate. Herein, recent advances in the direct growth of graphene on group‐IV semiconductors are reviewed, focusing mainly on the growth mechanism and initial growth behavior when graphene is synthesized on Si and Ge. Furthermore, recent progress in the device applications of graphene with Si and Ge are presented. Finally, perspectives for future research in graphene with a semiconductor are discussed.