Rice is a major source of cadmium (Cd) intake for Asian people. Indica rice usually accumulates more Cd in shoots and grains than Japonica rice. However, underlying genetic bases for differential Cd accumulation between Indica and Japonica rice are still unknown. In this study, we cloned a quantitative trait locus (QTL) grain Cd concentration on chromosome 7 (GCC7) responsible for differential grain Cd accumulation between two rice varieties by performing QTL analysis and map‐based cloning. We found that the two GCC7 alleles, GCC7PA64s and GCC793‐11, had different promoter activity of OsHMA3, leading to different OsHMA3 expression and different shoot and grain Cd concentrations. By analyzing the distribution of different haplotypes of GCC7 among diverse rice accessions, we discovered that the high and low Cd accumulation alleles, namely GCC793‐11 and GCC7PA64s, were preferentially distributed in Indica and Japonica rice, respectively. We further showed that the GCC7PA64s allele can be used to replace the GCC793‐11 allele in the super cultivar 93‐11 to reduce grain Cd concentration without adverse effect on agronomic traits. Our results thus reveal that the QTL GCC7 with sequence variation in the OsHMA3 promoter is an important determinant controlling differential grain Cd accumulation between Indica and Japonica rice.