Low phosphorus stress significantly impacts the development of maize kernels. In this study, the 082 maize genotype, which has high phosphorus use efficiency, and the Ye107 maize genotype, which has low phosphorus use efficiency, were utilized to construct an F2:3 population. QTL mapping was then employed to determine the genetic basis of differences in the maize kernel traits of the two parents in a low-phosphorus environment. This analysis revealed several major QTLs that control environmental impacts on kernel length, kernel width, kernel thickness, and kernel weight. These QTLs were detected in all three environments and were distributed on five genome segments of chromosomes 3, 5, 6, and 9. Most environmental insensitive QTLs had stable effects in all low-phosphorus environments, indicating that they may be broadly applicable to breeding efforts designed to generate germplasm with higher phosphorus use efficiency.