Large-scale Asian studies on this topic are lacking. We evaluated the CRC risk associated with family history in the Korean population. We analyzed the data of participants aged ≥40 years who underwent national cancer screening between 2013 and 2014. During a mean follow-up of 4.7 ± 0.8 years, 0.43% of the 292,467 participants with family history and 0.28% of the 1,169,868 participants without family history developed CRC. Participants with a family history in any FDR, parents only, and siblings only had a higher risk of CRC than those without family history; adjusted hazard ratios (HRs) were 1.53, 1.46, and 1.61, respectively. Participants with a family history comprising both parents and siblings had an even higher risk of CRC than those without a family history (HR, 2.34). The HRs for CRC in the 40–49, 50–59, 60–69, 70–79, and ≥80 age groups with family history were 1.72, 1.74, 1.50, 1.30, and 0.78, respectively (p < 0.001). A family history of CRC in any FDR and both parents and siblings was associated with an approximately 1.5- and 2.3-fold increased risk of CRC. The effect of family history was relatively greater in the younger than the older age group.