The current article aimed to characterize the spatial–temporal characteristics and the driving factors of disaster-induced grain yield loss. Taking the world’s largest agricultural country, China, as the research object, this study evaluated China’s disaster-induced grain yield loss from 1978 to 2019 and investigated the contribution of three driving factors, using a logarithmic mean Divisia index (LMDI) approach. The results showed that drought and flood were the main disasters causing grain yield losses in China. China’s grain disaster loss peaked in 2000 and then entered a declining period. In terms of spatial distribution, disaster-induced grain yield loss mainly concentrated in the regions including the Middle-Lower Yangtze region, the Huang-Huai-Hai region, and Northeast China. The gravity center of disaster-induced grain yield loss shifted northward in China. Disaster intensity contributed the most to change in disaster-induced grain yield loss, followed by yield loss density, and the grain planting area. Thus, addressing drought and flood disasters, and enhancing disaster prevention and reduction capacity are critical to reduce disaster-induced grain yield loss, especially in the Middle-Lower Yangtze, the Huang-Huai-Hai region, and Northeast China.