Enriching zinc (Zn) and selenium (Se) levels, while reducing cadmium (Cd) concentration in rice grains is of great benefit for human diet and health. Large natural variations in grain Zn, Se, and Cd concentrations in different rice accessions enable Zn/Se‐biofortification and Cd‐minimization through molecular breeding. Here, we report the development of new elite varieties by pyramiding major quantitative trait loci (QTLs) that significantly contribute to high Zn/Se and low Cd accumulation in grains. A chromosome segment substitution line CSSLGCC7 with the PA64s‐derived GCC7 allele in the 93‐11 background, exhibited steadily higher Mn and lower Cd concentrations in grains than those of 93‐11. This elite chromosome segment substitution line (CSSL) was used as the core breeding material to cross with CSSLs harboring other major QTLs for essential mineral elements, especially CSSLGZC6 for grain Zn concentration and CSSLGSC5 for grain Se concentration. The CSSLGCC7+GZC6 and CSSLGCC7+GSC5 exhibited lower Cd concentration with higher Zn and Se concentrations in grains, respectively. Our study thus provides elite materials for rice breeding targeting high Zn/Se and low Cd concentrations in grains.