Appropriate nitrogen (N) management system is essential for effective crop productivity and minimizing agricultural pollution. However, the underlying mechanistic understanding of how N fertilizer regulates crop yield via soil properties in soils with different fertilities remains unresolved. Here, we used a field experiment that spanned 3 cropping seasons to evaluate the grain yield (GY), aboveground biomass and N recovery efficiency (NRE) after treatment with five N fertilizer application rates (N0, N75, N112, N150, and N187) in soils with three levels of fertility. Our results indicated that the highest GY across low, moderate, and high fertility soils were 1.5 t hm-2 (N150), 4.9 t hm-2 (N187), and 5.4 t hm-2 (N112), respectively. The highest aboveground biomass and NRE were observed at N150 for all three levels of soil fertility, while only the N uptake by aboveground biomass of low and high fertility soils decreased at N187, confirming that excessive N fertilization results in a further decline in crop N uptake. The relationship between GY, NRE and N fertilizer application rates fit the unary quadratic polynomial model. To achieve a balance between grain production and environmental benefits in N fertilizer, appropriate N fertilizer rates were determined to be 97.5 kg hm-2, 140 kg hm-2 and 131 kg hm-2 for low, moderate and high fertility soils, respectively. Structural equation modeling suggested that GY was significant correlated with soil microbial biomass carbon (SMBC) and N directly in low fertility field, with SMBC directly in moderate fertility field, and via SOC and NO3–N in high fertility field. Therefore, a soil-based management strategy for N fertilizers could enhance food security while reducing agricultural N fertilizer inputs to mitigate environmental impacts.