Large quantities of fertilizers, mainly nitrogen (N), phosphorus (P), and potassium (K), are used in China's crop production; however, whether N, P, and K still limit yield under the current high input remains unknown. In addition, it is not clear whether magnesium (Mg) potentially limits crop production. To address these issues, a national survey of the soil nutrient status and farmers' fertilization practices was conducted in China to determine the soil N, P, K, and Mg status and current fertilization practices. In addition, various experiments, including optimized NPK (OPTNPK) and Mg fertilizer combined with OPTNPK (OPTNPK + Mg) in comparison with farmers' practices (FFP), were performed across China to identify the potential for reducing NPK application and the need for Mg application in crop production. The survey data from 2699 questionnaires indicated that (1) all soil samples were not deficient in NPK, but 73.0% were limited by Mg. (2) The average N, P (in P2O5), K (in K2O), and Mg (in MgO) balances were 422.6, 334.1, 326.3, and −22.6 kg ha−1, respectively. Only 15.7% of all farmers applied Mg fertilizer. (3) NPK fertilization can be reduced by an average of 28.8% without reducing the crop yield and quality. (4) For most crops, the application of Mg fertilizer increased the crop yield by 5.5–40.6% and improved the quality. However, different crops under different conditions had different responses to Mg fertilization. These results indicate that reduced NPK inputs and increased Mg inputs are equally important for the healthy development and sustainability of China's agriculture.