A review of automatic selection methods for machine learning algorithms and hyper-parameter values

Luo, Gang

doi:10.1007/s13721-016-0125-6

Cited by 277 publications

(176 citation statements)

References 43 publications

Supporting

Mentioning

172

Contrasting

Unclassified

Order By: Relevance

“…18 Along similar lines, automated machine learning (auto-ML) approaches the selection of a mapping function and its parameters as a machine learning task, allowing for selection and optimization of a mapping function or ensemble of functions with little user input. 16 With small datasets or excessively complex models, the mapping function may "overfit" the training data, learning properties inherent to the sample rather than relationships in the overall population; as a result, the generalizability, or out-of-sample accuracy, of the algorithm is diminished. 19 To limit overfitting, the training data may be divided into a training set, from which the algorithm sets its parameters, and a validation set, from which a preliminary estimate of the algorithm's generalizability is obtained ( Figure 1F).…”

Section: Key Pointsmentioning

confidence: 99%

Machine learning applications in epilepsy

2019

View full text Add to dashboard Cite

Machine learning leverages statistical and computer science principles to develop algorithms capable of improving performance through interpretation of data rather than through explicit instructions. Alongside widespread use in image recognition, language processing, and data mining, machine learning techniques have received increasing attention in medical applications, ranging from automated imaging analysis to disease forecasting. This review examines the parallel progress made in epilepsy, highlighting applications in automated seizure detection from electroencephalography (EEG), video, and kinetic data, automated imaging analysis and pre‐surgical planning, prediction of medication response, and prediction of medical and surgical outcomes using a wide variety of data sources. A brief overview of commonly used machine learning approaches, as well as challenges in further application of machine learning techniques in epilepsy, is also presented. With increasing computational capabilities, availability of effective machine learning algorithms, and accumulation of larger datasets, clinicians and researchers will increasingly benefit from familiarity with these techniques and the significant progress already made in their application in epilepsy.

show abstract

Section: Key Pointsmentioning

confidence: 99%

Machine learning applications in epilepsy

2019

View full text Add to dashboard Cite

show abstract

“…In this section, we briefly review existing automatic selection methods for both machine learning algorithms and hyperparameter values. A detailed review of existing automatic selection methods for algorithms and/or hyper-parameter values is provided in our papers [11,15].…”

Section: Review Of Existing Automatic Selection Methodsmentioning

confidence: 99%

Progressive sampling-based Bayesian optimization for efficient and automatic machine learning model selection

Zeng

Luo

2017

Health Inf Sci Syst

Self Cite

View full text Add to dashboard Cite

Purpose: Machine learning is broadly used for clinical data analysis. Before training a model, a machine learning algorithm must be selected. Also, the values of one or more model parameters termed hyper-parameters must be set. Selecting algorithms and hyper-parameter values requires advanced machine learning knowledge and many laborintensive manual iterations. To lower the bar to machine learning, miscellaneous automatic selection methods for algorithms and/or hyper-parameter values have been proposed. Existing automatic selection methods are inefficient on large data sets. This poses a challenge for using machine learning in the clinical big data era. Methods:To address the challenge, this paper presents progressive sampling-based Bayesian optimization, an efficient and automatic selection method for both algorithms and hyper-parameter values. Results:We report an implementation of the method. We show that compared to a state of the art automatic selection method, our method can significantly reduce search time, classification error rate, and standard deviation of error rate due to randomization.Conclusions: This is major progress towards enabling fast turnaround in identifying high-quality solutions required by many machine learning-based clinical data analysis tasks.

show abstract

“…For tasks such as scheduling machine learning model building jobs [59] and automatic machine learning model selection [10, 17, 18, 40, 68], meta-learning has been used before to predict model accuracy and building time. There, no constraint is put on the hyper-parameter value combinations selected for testing.…”

Section: Advanced Potential Uses Of Progress Indicatorsmentioning

confidence: 99%

Toward a Progress Indicator for Machine Learning Model Building and Data Mining Algorithm Execution

Luo

2017

SIGKDD Explor. Newsl.

Self Cite

View full text Add to dashboard Cite

For user-friendliness, many software systems offer progress indicators for long-duration tasks. A typical progress indicator continuously estimates the remaining task execution time as well as the portion of the task that has been finished. Building a machine learning model often takes a long time, but no existing machine learning software supplies a non-trivial progress indicator. Similarly, running a data mining algorithm often takes a long time, but no existing data mining software provides a nontrivial progress indicator. In this article, we consider the problem of offering progress indicators for machine learning model building and data mining algorithm execution. We discuss the goals and challenges intrinsic to this problem. Then we describe an initial framework for implementing such progress indicators and two advanced, potential uses of them, with the goal of inspiring future research on this topic.

show abstract

A review of automatic selection methods for machine learning algorithms and hyper-parameter values

Cited by 277 publications

References 43 publications

Machine learning applications in epilepsy

Machine learning applications in epilepsy

Progressive sampling-based Bayesian optimization for efficient and automatic machine learning model selection

Toward a Progress Indicator for Machine Learning Model Building and Data Mining Algorithm Execution

Contact Info

Product

Resources

About