Consumption of coffee, a popular beverage worldwide, has been associated with lower colorectal cancer (CRC) risk. Although CRC exhibits different biological characteristics by anatomical subsite, the possibly heterogeneous impact of coffee on CRC by anatomical subsite has remained unclear. Here, we conducted two case-control studies to examine the association between coffee consumption and CRC risk as well as risk by anatomic subsite among Japanese using data from the Hospital-based Epidemiological Research Program at Aichi Cancer Center I and II (HERPACC-I and II). Subjects were enrolled in HERPACC-I between 1988 and 2000 and in HERPACC-II between 2001 and 2005. Coffee consumption was measured with a self-administered questionnaire. A conditional logistic regression model was used to calculate odds ratios (ORs) of CRC with coffee consumption, adjusted for potential confounders of age, smoking, alcohol drinking, red meat intake, BMI, exercise, family history of CRC, and diabetes mellitus history. We estimated summary ORs by pooling study-specific ORs with a fixed effects model. In total, 2,696 CRC cases and 13,480 non-cancer outpatients as controls were included. Overall, compared to non-drinkers, ORs of less than 1 cup/day, 1-2 cups/day and 3 or more cups/day for CRC were 0.88 (95% CI: 0.77-1.00), 0.90 (95% CI: 0.80-1.01) and 0.78 (95% CI: 0.65-0.92), respectively (trend-p = 0.009). Subsite-specific analysis revealed a significant inverse linear trend between coffee consumption and distal colon cancer (p-trend = 0.048), and a tendency toward a lower risk of rectal cancer (p-trend = 0.068). These findings suggest that coffee consumption might impact the prevention of CRC, especially distal colon cancer.