Global Gridded Crop Model evaluation: benchmarking, skills, deficiencies and implications

Abstract

Abstract. Crop models are increasingly used to simulate crop yields at the global scale, but there so far is no general framework on how to assess model performance. We here evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that GGCMs show mixed skill in reproducing time-series correlations or spatial patterns at the different spatial scales. Generally, maize, wheat and soybean simulations of many GGCMs are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models. Yield variability can be well reproduced for most major producer countries by many GGCMS and for all countries by at least some. A comparison with gridded yield data and a statistical analysis of the effects of weather variability on yield variability shows that the ensemble of GGCMs can explain more of the yield variability than an ensemble of regression models for maize and soybean, but not for wheat and rice. We identify future research needs in global gridded crop modeling and for all individual crop modeling groups. In the absence of a purely observation-based benchmark for model evaluation, we propose that the best performing crop model per crop and region establishes the benchmark for all others, and modelers are encouraged to investigate how crop model performance can be increased. We make our evaluation system accessible to all crop modelers so that also other modeling groups can test their model performance against the reference data and the GGCMI benchmark.

Reconciling irrigated food production with environmental flows for Sustainable Development Goals implementation

“…Overall, Supplementary Fig. 4, but also the broader benchmarking evaluation in the context of Global Gridded Crop Model Intercomparison (GGCMI)6376 provide evidence for LPJmL's capability of representing most relevant mechanisms of climate-induced signals in observed yields, in particular those linked to water stress. This is key to explaining inter-annual yield variability and eventually feedbacks from water management on food production levels.…”

“…The LPJmL model has been validated extensively in terms of biogeochemical, ecological and hydrological processes for both natural and agricultural systems5960616263. It has the unique feature to simulate vegetation dynamics and the carbon and water cycle in a single consistent framework and therefore bridges categories between a global gridded crop model and a global hydrological model64.…”

Excessive rainfall leads to maize yield loss of a comparable magnitude to extreme drought in the United States

“…The simulation data of 12 global gridded crop models that participated in the Global Gridded Crop Model Intercomparison of the AgMIP were obtained from http://www.rdcep.org/research-projects/ggcmi (Müller et al, ) (hereafter the AgMIP global crop models). We used the historical simulations for maize without irrigation from 1981 to 2010, at 0.5° spatial resolution.…”

“…Most modeling efforts, for example, the participant models in the Agricultural Model Inter‐comparison and Improvement Project (AgMIP), focus on the temperature response and CO 2 effects on crop yield (Bassu et al, ; Deryng et al, ; Maiorano et al, ; Schauberger et al, ; Wang et al, ), whereas the precipitation response and the excessive rainfall impact have for some time not been in the focus (Lobell & Asseng, ; Rosenzweig, Tubiello, Goldberg, Mills, & Bloomfield, ; van der Velde, Tubiello, Vrieling, & Bouraoui, ). The extent that excessive rainfall adversely affects maize production in the United States is still largely unknown, especially when compared with the impact of drought (i.e., deficient rainfall), and how well current process‐based global gridded crop models (i.e., global crop models participated in the AgMIP, as opposed to point‐based models; Müller et al, ) simulate such impacts has not been evaluated.…”

Modelling predicts that soybean is poised to dominate crop production across Africa

“…Statistical crop models usually express the relationship between yield or yield components and weather parameters in a form of regression equations (e.g., Lobell & Burke, ), which are calibrated by using corresponding observed yield and weather data varying in time or space or in both domains. Process‐based crop models simulate the key processes of the soil–plant system, including crop development (phenology), biomass accumulation, yield, water, and nutrient uptake, while taking into account the effects of environmental stresses as well as plant responses to elevated atmospheric carbon dioxide concentrations (e.g., Müller et al, ). Fodor et al () reported that the vast majority of soybean‐related climate change impact studies using land suitability and/or crop models project positive changes for Sub‐Saharan Africa.…”