Automatic early stopping using cross validation: quantifying the criteria

Prechelt, Lutz

doi:10.1016/s0893-6080(98)00010-0

Cited by 778 publications

(417 citation statements)

References 7 publications

Supporting

Mentioning

406

Contrasting

Unclassified

Order By: Relevance

“…To help avoid overfitting for high dimensional problems (or more generally whenever N is large relative to I), early stopping (Morgan and Bourlard 1990;Prechelt 1998)-an implicit form of regularization commonly used in iteratively-trained machine learning methods-may be effective for MBCn. Early stopping involves terminating calibration of a learning algorithm, in this case MBCn, prior to convergence on the calibration sample.…”

Section: Discussionmentioning

confidence: 99%

Multivariate quantile mapping bias correction: an N-dimensional probability density function transform for climate model simulations of multiple variables

2017

View full text Add to dashboard Cite

Index (FWI) System, a complicated set of multivariate indices that characterizes the risk of wildfire, are then calculated and verified against observed values. Third, MBCn is used to correct biases in the spatial dependence structure of CanRCM4 precipitation fields. Results are compared against a univariate quantile mapping algorithm, which neglects the dependence between variables, and two multivariate bias correction algorithms, each of which corrects a different form of inter-variable correlation structure. MBCn outperforms these alternatives, often by a large margin, particularly for annual maxima of the FWI distribution and spatiotemporal autocorrelation of precipitation fields.

show abstract

Section: Discussionmentioning

confidence: 99%

Multivariate quantile mapping bias correction: an N-dimensional probability density function transform for climate model simulations of multiple variables

2017

View full text Add to dashboard Cite

show abstract

“…This phenomenon is called over-fitting. To avoid over-fitting, the process of cross-validation is used to estimate the quality of the ANNs prediction model (Lutz Prechelt, 1998).…”

Section: Evaluation Of the Predictive Performancementioning

confidence: 99%

Ensemble artificial neural networks applied to predict the key risk factors of hip bone fracture for elders

Liu

Cui

Chou

et al. 2015

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

Hip bone fracture is one of the most important causes of morbidity and mortality in the elder adults. It is necessary to establish a prediction model to provide suggestions for elders. A total of 725 subjects were involved, including 228 patients with first low-trauma hip fracture and 497 ages-, sex-, and living area-matched controls (215 from the same hospital and 282 from community). All the subjects were interviewed with the same questionnaire, and the answers of the interviewees were recorded to be the database. Three-layer back-propagation Artificial Neural Networks (ANN) models were applied for females and males separately in this study to predict the risk of hip bone fracture for elders. Furthermore, to improve the accuracies and the generalizations of the models, the ensemble ANNs method was applied. To understand variables contributions and find the important variables for predicting hip fracture, sensitivity analysis and connection weights approach were applied. In this study, three ANNs prediction models were tested with different architectures. With the fivefold crossvalidation method evaluating the performances, one of the three models turned out to be the best prediction model and achieved a big success of prediction. The best area under the receiver operating characteristic (ROC) curve and the accuracy of the prediction model are 0.91 ± 0.028 (mean ± SD) and 0.85 ± 0.029 for females, while for males are 0.99 ± 0.015 and 0.93 ± 0.020. With the method of sensitivity analysis and connection weights, input variables were ranked according to contributions/importance, and the top 10 variables show great proportion of contribution to predict hip fracture. The top 10 important variables causing hip fracture for both females and males are similar to our previous results got from logistic regression model and other related researches. In conclusion, ANNs has successfully been to establish prediction models for predicting the risk of hip bone fracture for both female and male elder adults respectively and identified the top 10 important variables from 74 input variables to predict hip bone fracture of elders. This study verified the performance of ANNs to be a highly complex prediction model. Sept 2014Dear Prof. R. Allen, I use the electronic version to send this manuscript to you. The manuscript title is: "Ensemble back-propagation neural networks for predicting the risk of hip bone fracture for elders in Taiwan". We are submitting this material for possible publication in "Biomedical Signal Processing and Control". This material has not been submitted for publication or published elsewhere in whole or part. We believe this manuscript represents an original and significant contribution to the field of "Neural networks for predicting the risk of hip bone fracture" and therefore would like to be considered for publication in "Original Articles". AbstractHip bone fracture is one of the most important causes of morbidity and mortality in the elder adults. It is necessary to establish a prediction mod...

show abstract

“…In [14] a number of different criteria for early stopping are discussed and it is suggested that allowing the condition to be biased towards the latter stages of the search will yield small improvements in generalization accuracy. This said however, if we delay too much we run the risk of overfitting once again.…”

Section: Test Set Accuracymentioning

confidence: 99%

“…− Early Stopping: Overfitting is avoided in the training of supervised Neural Networks by stopping the training when performance on a validation set starts to deteriorate [7,14]. Of these three options, the one that we explore here is Early Stopping.…”

Section: Introductionmentioning

confidence: 99%

Overfitting in Wrapper-Based Feature Subset Selection: The Harder You Try the Worse it Gets

Loughrey

Cunningham

Research and Development in Intelligent Systems XXI

View full text Add to dashboard Cite

Abstract.In Wrapper based feature selection, the more states that are visited during the search phase of the algorithm the greater the likelihood of finding a feature subset that has a high internal accuracy while generalizing poorly. When this occurs, we say that the algorithm has overfitted to the training data. We outline a set of experiments to show this and we introduce a modified genetic algorithm to address this overfitting problem by stopping the search before overfitting occurs. This new algorithm called GAWES (Genetic Algorithm With Early Stopping) reduces the level of overfitting and yields feature subsets that have a better generalization accuracy.

show abstract

Automatic early stopping using cross validation: quantifying the criteria

Cited by 778 publications

References 7 publications

Multivariate quantile mapping bias correction: an N-dimensional probability density function transform for climate model simulations of multiple variables

Multivariate quantile mapping bias correction: an N-dimensional probability density function transform for climate model simulations of multiple variables

Ensemble artificial neural networks applied to predict the key risk factors of hip bone fracture for elders

Overfitting in Wrapper-Based Feature Subset Selection: The Harder You Try the Worse it Gets

Contact Info

Product

Resources

About