How well do crop modeling groups predict wheat phenology, given calibration data from the target population?

Wallach, Daniel; Palosuo, Taru; Thorburn, Peter J.; Seidel, Sabine J.; Gourdain, Emmanuelle; Asseng, Senthold; Basso, Bruno; Buis, Samuel; Crout, N.M.J.; Dibari, Camilla; Dumont, Benjamin; Ferrise, Roberto; Gaiser, Thomas; Garcia, Cécile; Gayler, Sebastian; Ghahramani, Afshin; Hochman, Zvi; Hoek, Steven; Horan, Heidi; Hoogenboom, Gerrit; Huang, Mingxia; Jabloun, Mohamed; Jing, Qi; Justes, Éric; Kersebaum, Kurt Christian; Klosterhalfen, Anne; Launay, Marie; Luo, Qunying; Maestrini, Bernardo; Mielenz, Henrike; Moriondo, Marco; Zadeh, Hasti Nariman; Olesen, Jørgen E.; Poyda, Arne; Priesack, Eckart; Pullens, Johannes Wilhelmus Maria; Qian, Budong; Schütze, Niels; Shelia, Vakhtang; Souissi, Amir; Specka, Xenia; Srivastava, Amit Kumar; Stella, Tommaso; Streck, Thilo; Trombi, Giacomo; Wallor, Evelyn; Wang, Jing; Weber, Tobias; Weihermüller, Lutz; Wit, Allard de; Wöhling, Thomas; Xiao, Liujun; Zhao, Chuang; Zhu, Yan

doi:10.1101/708578

Cited by 14 publications

(23 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ensemble of users within ensemble of models) cannot be regarded as the solution to estimate uncertainty due to parameterization (Confalonieri et al, 2016). As well, different calibration techniques do not seem to be primarily responsible for differences in model performance (Wallach et al, 2020) and the contribution of the initialization to the uncertainty in SOC changes can be negligible compared to the uncertainty related to the model itself and simulated systems characteristics (Dimassi et al, 2018). As uncertainty could not be associated with any individual simulation, we focussed on the analysis of model residuals.…”

Section: Multi-model and Ensemble Assessmentmentioning

confidence: 99%

Ensemble modelling, uncertainty and robust predictions of organic carbon in long‐term bare‐fallow soils

Farina¹,

Sándor

Abdalla

et al. 2020

Global Change Biology

Self Cite

View full text Add to dashboard Cite

Simulation models represent soil organic carbon (SOC) dynamics in global carbon (C) cycle scenarios to support climate‐change studies. It is imperative to increase confidence in long‐term predictions of SOC dynamics by reducing the uncertainty in model estimates. We evaluated SOC simulated from an ensemble of 26 process‐based C models by comparing simulations to experimental data from seven long‐term bare‐fallow (vegetation‐free) plots at six sites: Denmark (two sites), France, Russia, Sweden and the United Kingdom. The decay of SOC in these plots has been monitored for decades since the last inputs of plant material, providing the opportunity to test decomposition without the continuous input of new organic material. The models were run independently over multi‐year simulation periods (from 28 to 80 years) in a blind test with no calibration (Bln) and with the following three calibration scenarios, each providing different levels of information and/or allowing different levels of model fitting: (a) calibrating decomposition parameters separately at each experimental site (Spe); (b) using a generic, knowledge‐based, parameterization applicable in the Central European region (Gen); and (c) using a combination of both (a) and (b) strategies (Mix). We addressed uncertainties from different modelling approaches with or without spin‐up initialization of SOC. Changes in the multi‐model median (MMM) of SOC were used as descriptors of the ensemble performance. On average across sites, Gen proved adequate in describing changes in SOC, with MMM equal to average SOC (and standard deviation) of 39.2 (±15.5) Mg C/ha compared to the observed mean of 36.0 (±19.7) Mg C/ha (last observed year), indicating sufficiently reliable SOC estimates. Moving to Mix (37.5 ± 16.7 Mg C/ha) and Spe (36.8 ± 19.8 Mg C/ha) provided only marginal gains in accuracy, but modellers would need to apply more knowledge and a greater calibration effort than in Gen, thereby limiting the wider applicability of models.

show abstract

Section: Multi-model and Ensemble Assessmentmentioning

confidence: 99%

Ensemble modelling, uncertainty and robust predictions of organic carbon in long‐term bare‐fallow soils

Farina¹,

Sándor

Abdalla

et al. 2020

Global Change Biology

Self Cite

View full text Add to dashboard Cite

show abstract

“…The results here show that phenology predictive performance for the calibration environments is significantly correlated with predictive performance for new environments. This was also found to be the case for a study evaluating phenology prediction by modeling groups based on phenology in French environments (Wallach et al, 2019) and suggests that in these cases, it may be worthwhile to use performance-weighted model ensembles. This may be due to the fact that in these studies, the calibration and evaluation environments were similar to one another.…”

Section: Model Averagingmentioning

confidence: 68%

“…The within-structure standard deviation here is 4.3 days, compared to a between-structure standard deviation (contribution of structure) of 11.9 days. The study based on French environments found a within-structure standard deviation of 5.6 days and a betweenstructure standard deviation of 8.0 days (Wallach et al, 2019). Confalonieri et al (2016) also found that the within-structure effect was in general, but not in all cases, smaller than the between-structure effect on variability.…”

Section: Sources Of Variabilitymentioning

confidence: 91%

“…It is of interest to compare the results here with those from a study structured like the present study (calibration and evaluation environments with similar characteristics, evaluation data not used for model development or tuning) but where the evaluation concerned prediction of two phenological stages of wheat in France, namely BBCH30 (equivalent to Z30) and BBCH55 (equivalent to Z55) (Wallach et al, 2019). To a large extent, the same modeling groups participated in both studies.…”

Section: Prediction Errormentioning

confidence: 96%

See 1 more Smart Citation

Multi-model evaluation of phenology prediction for wheat in Australia

Wallach

Palosuo

Thorburn

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

57Predicting wheat phenology is important for cultivar selection, for effective crop 58 management and provides a baseline for evaluating the effects of global change. Evaluating 59 how well crop phenology can be predicted is therefore of major interest. Twenty-eight wheat 60 modeling groups participated in this evaluation. Model predictions depend not only on model 61 structure but also on the parameter values. This study is thus an evaluation of modeling groups, 62 which choose the structure and fix or estimate the parameters, rather than an evaluation just of 63 model structures. Our target population was wheat fields in the major wheat growing regions 64 of Australia under current climatic conditions and with current local management practices. 65The environments used for calibration and for evaluation were both sampled from this same 66 target population. The calibration and evaluation environments had neither sites nor years in 67 common, so this is a rigorous evaluation of the ability of modeling groups to predict phenology 68 for new sites and weather conditions. Mean absolute error (MAE) for the evaluation 69 environments, averaged over predictions of three phenological stages and over modeling 70 groups, was 9 days, with a range from 6 to 20 days. Predictions using the multi-modeling group 71 mean and median had prediction errors nearly as small as the best modeling group. For a given 72 modeling group, MAE for the evaluation environments was significantly correlated with MAE 73 for the calibration environments, which suggests that it would be of interest to test ensemble 74 predictors that weight individual modeling groups based on performance for the calibration 75 data. About two thirds of the modeling groups performed better than a simple but relevant 76 benchmark, which predicts phenology by assuming a constant temperature sum for each 77 development stage. The added complexity of crop models beyond just the effect of temperature 78 was thus justified in most cases. Finally, there was substantial variability between modeling 79 groups using the same model structure, which implies that model improvement could be 80 4 achieved not only by improving model structure, but also by improving parameter values, and 81 in particular by improving calibration techniques. 82 uncertainty 84 85 5 A second aspect of evaluation that must be specified is the modeling group or groups 131 that are being evaluated. We reserve the term "model" specifically for model structure, i.e. the 132 model equations, while modeling group determines both the model structure and the parameter 133 values, which are chosen or estimated by the group running the model. It is clear that predictions 134

show abstract

“…These can be stand-alone temperature (and sometimes photoperiod) driven phenology models, such as DD10 for rice [17,18] or more complex crop growth simulation models such as APSIM [19] or ORYZA [20]. These models are particularly useful to assess the expected crop response to changes in the environment or management, but in practice they do not always perform well due to the insufficient empirical data available for calibration [21], and a limited ability to capture the effects of extreme weather events on crop phenology [22][23][24]. Moreover, the use of these models can be overly simplistic if differences in the environment, variety and other crop managements are insufficiently represented [15,25].…”

Section: Introductionmentioning

confidence: 99%

Satellite-Based Observations Reveal Effects of Weather Variation on Rice Phenology

2020

View full text Add to dashboard Cite

Obtaining detailed data on the spatio-temporal variation in crop phenology is critical to increasing our understanding of agro-ecosystem function, such as their response to weather variation and climate change. It is challenging to collect such data over large areas through field observations. The use of satellite remote sensing data has made phenology data collection easier, although the quality and the utility of such data to understand agro-ecosystem function have not been widely studied. Here, we evaluated satellite data-based estimates of rice phenological stages in California, USA by comparing them with survey data and with predictions by a temperature-driven phenology model. We then used the satellite data-based estimates to quantify the crop phenological response to changes in weather. We used time-series of MODIS satellite data and PhenoRice, a rule-based rice phenology detection algorithm, to determine annual planting, heading and harvest dates of paddy rice in California between 2002 and 2017. At the state level, our satellite-based estimates of rice phenology were very similar to the official survey data, particularly for planting and harvest dates (RMSE = 3.8–4.0 days). Satellite based observations were also similar to predictions by the DD10 temperature-driven phenology model. We analyzed how the timing of these phenological stages varied with concurrent temperature and precipitation over this 16-year time period. We found that planting was earlier in warm springs (−1.4 days °C−1 for mean temperature between mid-April and mid-May) and later in wet years (5.3 days 100 mm-1 for total precipitation from March to April). Higher mean temperature during the pre-heading period of the growing season advanced heading by 2.9 days °C−1 and shortened duration from planting to heading by 1.9 days °C−1. The entire growing season was reduced by 3.2 days °C−1 because of the increased temperature during the rice season. Our findings confirm that satellite data can be an effective way to estimate variations in rice phenology and can provide critical information that can be used to improve understanding of agricultural responses to weather variation.

show abstract

How well do crop modeling groups predict wheat phenology, given calibration data from the target population?

Cited by 14 publications

References 38 publications

Ensemble modelling, uncertainty and robust predictions of organic carbon in long‐term bare‐fallow soils

Ensemble modelling, uncertainty and robust predictions of organic carbon in long‐term bare‐fallow soils

Multi-model evaluation of phenology prediction for wheat in Australia

Satellite-Based Observations Reveal Effects of Weather Variation on Rice Phenology

Contact Info

Product

Resources

About