Bagged ensembles with tunable parameters

Pham, Hieu; Ólafsson, Sigurđur

doi:10.1111/coin.12198

Cited by 31 publications

(24 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Studies show that a single machine learning model can be outperformed by a “committee” of individual models, which is called a machine learning ensemble ( Zhang and Ma, 2012 ). Ensemble learning is proved to be effective as it can reduce bias, variance, or both and is able to better capture the underlying distribution of the data in order to make better predictions, if the base learners are diverse enough ( Dietterich, 2000 ; Pham and Olafsson, 2019a ; Pham and Olafsson, 2019b ; Shahhosseini et al., 2019a ; Shahhosseini et al., 2019b ). The usage of ensemble learning in ecological problems is becoming more widespread; for instance, bagging and specifically random forest ( Vincenzi et al., 2011 ; Mutanga et al., 2012 ; Fukuda et al., 2013 ; Jeong et al., 2016 ), boosting ( De'ath, 2007 ; Heremans et al., 2015 ; Belayneh et al., 2016 ; Stas et al., 2016 ; Sajedi-Hosseini et al., 2018 ), and stacking ( Conţiu and Groza, 2016 ; Cai et al., 2017 ; Shahhosseini et al., 2019a ), are some of the ensemble learning applications in agriculture.…”

Section: Introductionmentioning

confidence: 99%

Forecasting Corn Yield With Machine Learning Ensembles

2020

View full text Add to dashboard Cite

The emergence of new technologies to synthesize and analyze big data with highperformance computing has increased our capacity to more accurately predict crop yields. Recent research has shown that machine learning (ML) can provide reasonable predictions faster and with higher flexibility compared to simulation crop modeling. However, a single machine learning model can be outperformed by a "committee" of models (machine learning ensembles) that can reduce prediction bias, variance, or both and is able to better capture the underlying distribution of the data. Yet, there are many aspects to be investigated with regard to prediction accuracy, time of the prediction, and scale. The earlier the prediction during the growing season the better, but this has not been thoroughly investigated as previous studies considered all data available to predict yields. This paper provides a machine leaning based framework to forecast corn yields in three US Corn Belt states (Illinois, Indiana, and Iowa) considering complete and partial inseason weather knowledge. Several ensemble models are designed using blocked sequential procedure to generate out-of-bag predictions. The forecasts are made in county-level scale and aggregated for agricultural district and state level scales. Results show that the proposed optimized weighted ensemble and the average ensemble are the most precise models with RRMSE of 9.5%. Stacked LASSO makes the least biased predictions (MBE of 53 kg/ha), while other ensemble models also outperformed the base learners in terms of bias. On the contrary, although random k-fold cross-validation is replaced by blocked sequential procedure, it is shown that stacked ensembles perform not as good as weighted ensemble models for time series data sets as they require the data to be non-IID to perform favorably. Comparing our proposed model forecasts with the literature demonstrates the acceptable performance of forecasts made by our proposed ensemble model. Results from the scenario of having partial in-season weather knowledge reveals that decent yield forecasts with RRMSE of 9.2% can be made as early as June 1 st. Moreover, it was shown that the proposed model performed better than individual models and benchmark ensembles at agricultural district and statelevel scales as well as county-level scale. To find the marginal effect of each input feature on the forecasts made by the proposed ensemble model, a methodology is suggested that is the basis for finding feature importance for the ensemble model. The findings

show abstract

Section: Introductionmentioning

confidence: 99%

Forecasting Corn Yield With Machine Learning Ensembles

2020

View full text Add to dashboard Cite

show abstract

“…The techniques used can range from classical statistical techniques to Machine Learning (ML) algorithms. The latter have enjoyed wide applications in various ecological classification problems and predictive modeling (Rumpf et al 2010, Shekoofa et al 2014, Crane-Droesch 2018, Karimzadeh and Olafsson 2019, Pham and Olafsson 2019a, 2019b because of their adeptness to deal with nonlinear relationships, high-order interactions and non-normal data (De'ath and Fabricius 2000). Such methods include regularized regressions (Hoerl and Kennard 1970, Tibshirani 1996, Zou and Hastie 2005, tree-based models (Shekoofa et al 2014), Support Vector Machines (Basak et al 2007, Karimi et al 2008, Neural Networks (Liu et al 2001, Crane-Droesch 2018, Khaki and Khalilzadeh 2019, Khaki and Wang 2019 and others.…”

Section: Introductionmentioning

confidence: 99%

Maize yield and nitrate loss prediction with machine learning algorithms

Shahhosseini

Martinez-Feria

et al. 2019

Environ. Res. Lett.

152

View full text Add to dashboard Cite

Pre-growing season prediction of crop production outcomes such as grain yields and nitrogen (N) losses can provide insights to farmers and agronomists to make decisions. Simulation crop models can assist in scenario planning, but their use is limited because of data requirements and long runtimes. Thus, there is a need for more computationally expedient approaches to scale up predictions. We evaluated the potential of four machine learning (ML) algorithms (LASSO Regression, Ridge Regression, random forests, Extreme Gradient Boosting, and their ensembles) as meta-models for a cropping systems simulator (APSIM) to inform future decision support tool development. We asked:(1) How well do ML meta-models predict maize yield and N losses using pre-season information? (2) How many data are needed to train ML algorithms to achieve acceptable predictions? (3) Which input data variables are most important for accurate prediction? And (4) do ensembles of ML meta-models improve prediction? The simulated dataset included more than three million data including genotype, environment and management scenarios. XGBoost was the most accurate ML model in predicting yields with a relative mean square error (RRMSE) of 13.5%, and Random forests most accurately predicted N loss at planting time, with a RRMSE of 54%. ML meta-models reasonably reproduced simulated maize yields using the information available at planting, but not N loss. They also differed in their sensitivities to the size of the training dataset. Across all ML models, yield prediction error decreased by 10%-40% as the training dataset increased from 0.5 to 1.8 million data points, whereas N loss prediction error showed no consistent pattern. ML models also differed in their sensitivities to input variables (weather, soil properties, management, initial conditions), thus depending on the data availability researchers may use a different ML model. Modest prediction improvements resulted from ML ensembles. These results can help accelerate progress in coupling simulation models and ML toward developing dynamic decision support tools for pre-season management.

show abstract

“…Whereas boosting models is used to reduce bias. Each strategy possesses their strength and weaknesses, and finding the optimal balance between the two remains a challenging problem [18], [19].…”

Section: Methodsmentioning

confidence: 99%

Optimizing Ensemble Weights for Machine Learning Models: A Case Study for Housing Price Prediction

Shahhosseini

Pham

2019

Springer Proceedings in Business and Economics

Self Cite

View full text Add to dashboard Cite

Designing ensemble learners has been recognized as one of the significant trends in the field of data knowledge especially in data science competitions. Building models that are able to outperform all individual models in terms of bias, which is the error due to the difference in the average model predictions and actual values, and variance, which is the variability of model predictions, has been the main goal of the studies in this area. An optimization model has been proposed in this paper to design ensembles that try to minimize bias and variance of predictions. Focusing on service sciences, two well-known housing datasets have been selected as case studies: Boston housing and Ames housing. The results demonstrate that our designed ensembles can be very competitive in predicting the house prices in both Boston and Ames datasets.

show abstract

Bagged ensembles with tunable parameters

Cited by 31 publications

References 38 publications

Forecasting Corn Yield With Machine Learning Ensembles

Forecasting Corn Yield With Machine Learning Ensembles

Maize yield and nitrate loss prediction with machine learning algorithms

Optimizing Ensemble Weights for Machine Learning Models: A Case Study for Housing Price Prediction

Contact Info

Product

Resources

About