Based on panel data for the inputs and outputs of the three major staple grains (rice, wheat and corn) in China from 2000 to 2020, we calculated the cost efficiency using the stochastic frontier cost function model and examined the effects of cost changes for the three major grains to explore the sources of cost increases for grain production. On this basis, the impact of the input factor structures on the technical efficiency of the three food grains was further analyzed under the conditions of price increases. We found that the labor prices and production costs showed the same trends of changes. Compared to 2000, the labor prices in 2020 increased 7.33-fold and the technical efficiency values for the three grains were all close to 0.9 (0.8689, 0.8912 and 0.8451). An efficiency decomposition showed that the adjustment effect of labor prices was the main factor in cost increases, but the effects of technological progress and efficiency improvement could effectively reduce the costs of grain production (the largest average value for technological progress was for rice at 0.4569). In comparison, the effects of technological progress on cost reduction were more obvious. By analyzing the influence of input factor structures on technical efficiency, it was found that the influence of different input factor structures on technical efficiency was heterogeneous among the different grains. This paper puts forward the following policy recommendations: first, improve the level of mechanization by developing social services to reduce the dependence on labor; secondly, promote the construction of agricultural informatization, such as accelerating the research and development of intelligent agricultural machinery and promoting the transformation of traditional agriculture to intelligent agriculture; finally, promote the marketization of land element price through land trusteeship to reduce the land transfer price.