Micronutrient deficiencies, and especially zinc (Zn) deficiency, pose serious health problems to people who mainly depend on cereal-based diets. Here, we performed a genome-wide association study (GWAS) to detect the genetic basis of the Zn accumulation in wheat (Triticum aestivum L.) grains with a diversity panel of 207 bread wheat varieties. To uncover authentic quantitative trait loci (QTL) controlling Zn accumulation, the varieties were planted in three locations. In total, 29 unique loci associated with Zn grain accumulation were identified. Notably, seven non-redundant loci located on chromosomes 1B, 3B, 3D, 4A, 5A, 5B, and 7A, were detected at least in two environments. Of these quantitative trait loci (QTL), six coincided with known QTL or genes, whereas the highest effect QTL on chromosome 3D identified in this study was not reported previously. Searches of public databases revealed that the seven identified QTL coincided with seven putative candidate genes linked to Zn accumulation. Among these seven genes, NAC domain-containing protein gene (TraesCS3D02G078500) linked with the most significant single nucleotide polymorphism (SNP) AX-94729264 on chromosome 3D was relevant to metal accumulation in wheat grains. Results of this study provide new insights into the genetic architecture of Zn accumulation in wheat grains.