Fast robust estimation of prediction error based on resampling

Khan, Jafar A; Aelst, Stefan Van; Zamar, Ruben H.

doi:10.1016/j.csda.2010.01.031

Cited by 30 publications

(16 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Newlands et al, 2014). The selected predictors were then subjected to a robust cross validation (RCV) scheme (Khan et al, 2010) to finalise the predictors and coefficients of each yield model. The RCV process further stabilized the model by eliminating any false predictors selected from contaminated data (Khan et al, 2010).…”

Section: Iccyf and Its Statistical Algorithmsmentioning

confidence: 99%

“…The selected predictors were then subjected to a robust cross validation (RCV) scheme (Khan et al, 2010) to finalise the predictors and coefficients of each yield model. The RCV process further stabilized the model by eliminating any false predictors selected from contaminated data (Khan et al, 2010). Based on Bornn and Zidek (2012), incorporating spatial correlation among the yield models of neighbouring CARs increases the individual model's prediction power and stabilizes the model performance.…”

Section: Iccyf and Its Statistical Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of the Integrated Canadian Crop Yield Forecaster (ICCYF) model for in-season prediction of crop yield across the Canadian agricultural landscape

Chipanshi

Zhang

Kouadio

et al. 2015

Agricultural and Forest Meteorology

133

View full text Add to dashboard Cite

a b s t r a c tEarly warning information on crop yield and production are very crucial for both farmers and decisionmakers. In this study, we assess the skill and the reliability of the Integrated Canadian Crop Yield Forecaster (ICCYF), a regional crop yield forecasting tool, at different temporal (i.e. 1-3 months before harvest) and spatial (i.e. census agricultural region -CAR, provincial and national) scales across Canada. A distinct feature of the ICCYF is that it generates in-season yield forecasts well before the end of the growing season and provides a probability distribution of the forecasted yields. The ICCYF integrates climate, remote sensing derived vegetation indices, soil and crop information through a physical process-based soil water budget model and statistical algorithms. The model was evaluated against yield survey data of spring wheat, barley and canola during the 1987-2012 period. Our results showed that the ICCYF performance exhibited a strong spatial pattern at both CAR and provincial scales. Model performance was better from regions with a good coverage of climate stations and a high percentage of cropped area. On average, the model coefficient of determination at CAR level was 66%, 51% and 67%, for spring wheat, barley and canola, respectively. Skilful forecasts (i.e. model efficiency index > 0) were achieved in 70% of the CARs for spring wheat and canola, and 43% for barley (low values observed in CAR with small harvested area). At the provincial scale, the mean absolute percentage errors (MAPE) of the September forecasts ranged from 7% to 16%, 7% to14%, and 6% to 14% for spring wheat, barley and canola, respectively. For forecasts at the national scale, MAPE values (i.e. 8%, 5% and 9% for the three respective crops) were considerably smaller than the corresponding historical coefficients of variation (i.e. 17%, 10% and 17% for the three crops). Overall, the ICCYF performed better for spring wheat than for canola and barley at all the three spatial scales. Skilful forecasts were achieved by mid-August, giving a lead time of about 1 month before harvest and about 3-4 months before the final release of official survey results. As such, the ICCYF could be used as a complementary tool for the traditional survey method, especially in areas where it is not practical to conduct such surveys.Crown

show abstract

Section: Iccyf and Its Statistical Algorithmsmentioning

confidence: 99%

Section: Iccyf and Its Statistical Algorithmsmentioning

confidence: 99%

Evaluation of the Integrated Canadian Crop Yield Forecaster (ICCYF) model for in-season prediction of crop yield across the Canadian agricultural landscape

Chipanshi

Zhang

Kouadio

et al. 2015

Agricultural and Forest Meteorology

133

View full text Add to dashboard Cite

show abstract

“…Cross-validation is an estimator widely used to evaluate prediction errors (Borra and Di Ciaccio, 2010;Khan et al, 2010). In k-fold cross-validation (see, e.g., Hastie et al (2009)), the overall available data is randomly partitioned into k sets of equal size: each of the k sets is used once to measure the OOS forecast accuracy and the other k − 1 sets are used to build the model.…”

Section: Blocked Cross-validationmentioning

confidence: 99%

On the usefulness of cross-validation for directional forecast evaluation

Bergmeir

Costantini

Benítez

2014

Computational Statistics & Data Analysis

View full text Add to dashboard Cite

The usefulness of a predictor evaluation framework which combines a blocked cross-validation scheme with directional accuracy measures is investigated. The advantage of using a blocked cross-validation scheme with respect to the standard out-of-sample procedure is that cross-validation yields more precise error estimates of the prediction error since it makes full use of the data. In order to quantify the gain in precision when directional accuracy measures are considered, a Monte Carlo analysis using univariate and multivariate models is provided. The experiments indicate that more precise estimates are obtained with the blocked cross-validation procedure. An application is carried out on forecasting UK interest rate for illustration purposes. The results show that in such a situation with small samples the crossvalidation scheme may have considerable advantages over the standard out-of-sample evaluation procedure as it may help to overcome problems induced by the limited information the directional accuracy measures contain due to their binary nature.

show abstract

“…The RLARS was applied to account for heteroscedasticity and outliers in the historical data (i.e., model training/calibration). The selected predictors were then subjected to a Robust Cross Validation (RCV) scheme [39] to further stabilized the model by eliminating any false predictors selected from contaminated data [39]. Then, the Bayesian statistical approach as described by Bornn and Zidek [40] for the spatial correlation analysis among the neighboring spatial units was applied.…”

Section: Overview Of the Integrated Canadian Crop Yield Forecaster (Imentioning

confidence: 99%

Assessing the Performance of MODIS NDVI and EVI for Seasonal Crop Yield Forecasting at the Ecodistrict Scale

et al. 2014

View full text Add to dashboard Cite

Crop yield forecasting plays a vital role in coping with the challenges of the impacts of climate change on agriculture. Improvements in the timeliness and accuracy of yield forecasting by incorporating near real-time remote sensing data and the use of sophisticated statistical methods can improve our capacity to respond effectively to these challenges. The objectives of this study were (i) to investigate the use of derived vegetation indices for the yield forecasting of spring wheat (Triticum aestivum L.) from the Moderate resolution Imaging Spectroradiometer (MODIS) at the ecodistrict scale across Western Canada with the Integrated Canadian Crop Yield Forecaster (ICCYF); and (ii) to compare the ICCYF-model based forecasts and their accuracy across two spatial scales-the ecodistrict and Census Agricultural Region (CAR), namely in CAR with previously reported Remote Sens. 2014, 6 10194 ICCYF weak performance. Ecodistricts are areas with distinct climate, soil, landscape and ecological aspects, whereas CARs are census-based/statistically-delineated areas. Agroclimate variables combined respectively with MODIS-NDVI and MODIS-EVI indices were used as inputs for the in-season yield forecasting of spring wheat during the 2000-2010 period. Regression models were built based on a procedure of a leave-one-year-out. The results showed that both agroclimate + MODIS-NDVI and agroclimate + MODIS-EVI performed equally well predicting spring wheat yield at the ECD scale. The mean absolute error percentages (MAPE) of the models selected from both the two data sets ranged from 2% to 33% over the study period. The model efficiency index (MEI) varied between −1.1 and 0.99 and −1.8 and 0.99, respectively for the agroclimate + MODIS-NDVI and agroclimate + MODIS-EVI data sets. Moreover, significant improvement in forecasting skill (with decreasing MAPE of 40% and 5 times increasing MEI, on average) was obtained at the finer, ecodistrict spatial scale, compared to the coarser CAR scale. Forecast models need to consider the distribution of extreme values of predictor variables to improve the selection of remote sensing indices. Our findings indicate that statistical-based forecasting error could be significantly reduced by making use of MODIS-EVI and NDVI indices at different times in the crop growing season and within different sub-regions.

show abstract

Fast robust estimation of prediction error based on resampling

Cited by 30 publications

References 29 publications

Evaluation of the Integrated Canadian Crop Yield Forecaster (ICCYF) model for in-season prediction of crop yield across the Canadian agricultural landscape

Evaluation of the Integrated Canadian Crop Yield Forecaster (ICCYF) model for in-season prediction of crop yield across the Canadian agricultural landscape

On the usefulness of cross-validation for directional forecast evaluation

Assessing the Performance of MODIS NDVI and EVI for Seasonal Crop Yield Forecasting at the Ecodistrict Scale

Contact Info

Product

Resources

About