Non-grain production of cropland (NGPCL) is a common phenomenon in the process of rapid urbanization in order to meet the diversified food demand and prosperity of the rural economy. However, excessive NGPCL will threaten grain production. How to control the moderate development of NGPCL in order to achieve the balance between food security and rural development has become a salient issue. In this study, we constructed a framework to measure NGPCL, revealed the spatial–temporal pattern of NGPCL, and then analyzed its influencing factors from the perspective of the human–land relationship. The results indicate that, firstly, the overall degree of NGPCL in China experienced an increase from 0.44 to 0.51, while the gap among cities was consistently enlarging, with the range value increasing from 0.74 to 0.91. Secondly, the spatial pattern of NGPCL was high in the northwest and southeast, and low in the northeast and central regions. The southern economic developed area exhibited the highest increase, while the Inner Mongolia, northwest China, and traditional agricultural areas experienced a decreasing trend in NGPCL. Thirdly, the spatial agglomeration of NGPCL has been intensified, with the Gansu–Xinjiang Desert plateau, southeast coastal economic belt, and urban agglomeration areas exhibiting a “high–high” agglomeration, while the traditional agricultural areas exhibited “low–low” agglomeration. Fourthly, NGPCL is positively correlated with the urbanization rate, land fragmentation, landscape diversity, land price, and grain production policy, while it is negatively linked with the agricultural employment rate, agricultural machinery level, and cultivated land per capita. The findings of this research are not only deepen the understanding of NGPCL, but are also of great significance for policy makers in order to propose targeted control measures.