Performance of the MARS-crop yield forecasting system for the European Union: Assessing accuracy, in-season, and year-to-year improvements from 1993 to 2015

Abstract

19,980 crop yield forecasts have been published for the European Union (EU) Member States (MS) during 1993–2015 using the MARS-Crop Yield Forecasting System (MCYFS). We assess the performance of these forecasts for soft wheat, durum wheat, grain maize, rapeseed, sunflower, potato and sugar beet, and sought to answer three questions. First, how good has the system performed? This was investigated by calculating several accuracy indicators (e.g. the mean absolute percentage error, MAPE) for the first forecasts during a season, forecasts one month pre-harvest, and the end-of-campaign (EOC) forecasts during 2006–2015 using reported yields. Second, do forecasts improve during the season? This was evaluated by comparing the accuracy of the first, the pre-harvest, and the EOC forecasts. Third, have forecasts systematically improved year-to-year? This was quantified by testing whether linear models fitted to the median of the national level absolute relative forecast errors for each crop at EU-12 (EU-27) level from 1993 to 2015 (2006–2015) were characterized by significant negative slopes. Encouragingly, the lowest median MAPE across all crops is obtained for Europe's largest producer, France, equalling 3.73%. Similarly, the highest median MAPE is obtained for Portugal, at 14.37%. Forecasts generally underestimated reported yields, with a systematic underestimation across all MS for soft wheat, rapeseed and sugar beet forecasts. Forecasts generally improve during the growing season; both the forecast error and its variability tend to progressively decrease. This is the case for the cereals, and to a lesser extent for the tuber crops, while seasonal forecast improvements are lower for the oilseed crops. The median EU-12 yield forecasts for rapeseed, potato and sugar beet have significantly (p-value < 0.05) improved from 1993 to 2015. No evidence was found for improvements for the other crops, neither was there any significant improvement in any of the crop forecasts from 2006 to 2015, evaluated at EU-27 level. In the early years of the MCYFS, most of the yield time series were characterized by strong trends; nowadays yield growth has slowed or even plateaued in several MS. In addition, an increased volatility in yield statistics is observed, and while crop yield forecasts tend to improve in a given year, in recent years, there is no evidence of structural improvements that carry-over from year-to-year. This underlines that renewed efforts to improve operational crop yield forecasting are needed, especially in the light of the increasingly variable and occasionally unprecedented climatic conditions impacting the EU's crop production systems.

Multicriteria Prediction and Simulation of Winter Wheat Yield Using Extended Qualitative and Quantitative Data Based on Artificial Neural Networks

“…Table 4 presents the error values of the ex post measures for all created neural models. The most commonly used indicators characterizing prediction error values include MAPE [10,34,[38][39][40][41][42]. The lowest MAPE values were obtained for the QQWW31_5 neural model based on the MLP network of the 23:38-16-8-1:1 structure, which was 6.63%.…”

Modeling the Contribution of Crops to Nitrogen Pollution in the Yangtze River

“…18,19,27,35,36 WOFOST has been applied to simulate the production of the main annual crops over Europe 28,44 and China. 45 It also has been evaluated against experimental information in previous studies. 31,46 However, some incomplete data sets that were used to produce model inputs may introduce uncertainties in the simulation of WOFOST for the Yangtze basin.…”

Lower air pollution during COVID-19 lock-down: improving models and methods estimating ozone impacts on crops

“…However, crop growth models used for operational agronomic analyses do not include O 3 impacts. For example, the WOFOST model, currently used by the European Commission Joint Research Centre to provide operational analysis of crop growth development and yield forecasts [25], does not explicitly consider the effect of O 3 on crop phenology and growth. In spite of this, the yield forecasts for which it is used can usually achieve an accuracy within ±3% Figure 3.…”