Wheat (Triticum aestivum L.) is one of the major staple food crops worldwide. Despite efforts in improving wheat quality, micronutrient levels are still below the optimal range for human nutrition. In particular, zinc (Zn) deficiency is a widespread problem in human nutrition in countries relying mainly on a cereal diet; hence improving Zn accumulation in grains is an imperative need. This study was designed to understand the genetic architecture of Zn grain concentrations in wheat grains. We performed a genome-wide association study (GWAS) for grain Zn concentrations in 369 European wheat genotypes, using field data from 3 years. The complete wheat panel was genotyped by high-density arrays of single nucleotide polymorphic (SNP) markers (90k iSELECT Infinium and 35k Affymetrix arrays) resulting in 15,523 polymorphic markers. Additionally, a subpanel of 183 genotypes was analyzed with a novel 135k Affymetrix marker array resulting in 28,710 polymorphic SNPs for high-resolution mapping of the potential genomic regions. The mean grain Zn concentration of the genotypes ranged from 25.05–52.67 μg g-1 dry weight across years with a moderate heritability value. Notably, 40 marker-trait associations (MTAs) were detected in the complete panel of varieties on chromosomes 2A, 3A, 3B, 4A, 4D, 5A, 5B, 5D, 6D, 7A, 7B, and 7D. The number of MTAs in the subpanel was increased to 161 MTAs whereas the most significant and consistent associations were located on chromosomes 3B (723,504,241–723,611,488 bp) and 5A (462,763,758–466,582,184 bp) having major effects. These genomic regions include newly identified putative candidate genes, which are related to Zn uptake and transport or represent bZIP and mitogen-activated protein kinase genes. These findings provide the basis for understanding the genetic background of Zn concentration in wheat grains that in turn may help breeders to select high Zn-containing genotypes to improve human health and grain quality.