The online performance estimation framework: heterogeneous ensemble learning for data streams

Rijn, Jan N. van; Holmes, Geoffrey; Pfahringer, Bernhard; Vanschoren, Joaquin

doi:10.1007/s10994-017-5686-9

Cited by 101 publications

(78 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Newbold and Granger (1974) use this approach for combining forecasters models. More recently, van Rijn et al (2018) proposed a method for data streams classification. As opposed to fusing experts, they select the best recent performing one to classify the next observation.…”

Section: Windowing Strategies For Expert Combinationmentioning

confidence: 99%

See 1 more Smart Citation

Arbitrage of forecasting experts

et al. 2018

View full text Add to dashboard Cite

Forecasting is an important task across several domains. Its generalised interest is related to the uncertainty and complex evolving structure of time series. Forecasting methods are typically designed to cope with temporal dependencies among observations, but it is widely accepted that none is universally applicable. Therefore, a common solution to these tasks is to combine the opinion of a diverse set of forecasts. In this paper we present an approach based on arbitrating, in which several forecasting models are dynamically combined to obtain predictions. Arbitrating is a metalearning approach that combines the output of experts according to predictions of the loss that they will incur. We present an approach for retrieving out-of-bag predictions that significantly improves its data efficiency. Finally, since diversity is a fundamental component in ensemble methods, we propose a method for explicitly handling the inter-dependence between experts when aggregating their predictions. Results from extensive empirical experiments provide evidence of the method's competitiveness relative to state of the art approaches. The proposed method is publicly available in a software package.

show abstract

Section: Windowing Strategies For Expert Combinationmentioning

confidence: 99%

“…Similar to WindowLoss, but selects the best expert in the last λ observations for prediction. van Rijn et al (2018) showed its competitiveness using streaming data; AEC:…”

Section: Ensemble Setup and Baselinesmentioning

confidence: 99%

Arbitrage of forecasting experts

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Few concepts for automated algorithm selection on streaming data exist so far, both for supervised (see, e.g., van Rijn et al, 2014van Rijn et al, , 2018 and unsupervised learning algorithms. In unsupervised learning, stream clustering is a very active research field.…”

Section: Performance Measuresmentioning

confidence: 99%

Automated Algorithm Selection: Survey and Perspectives

Kerschke

Hoos

Neumann

et al. 2019

Evolutionary Computation

317

184

View full text Add to dashboard Cite

It has long been observed that for practically any computational problem that has been intensely studied, different instances are best solved using different algorithms. This is particularly pronounced for computationally hard problems, where in most cases, no single algorithm defines the state of the art; instead, there is a set of algorithms with complementary strengths. This performance complementarity can be exploited in various ways, one of which is based on the idea of selecting, from a set of given algorithms, for each problem instance to be solved the one expected to perform best. The task of automatically selecting an algorithm from a given set is known as the per-instance algorithm selection problem and has been intensely studied over the past 15 years, leading to major improvements in the state of the art in solving a growing number of discrete combinatorial problems, including propositional satisfiability and AI planning. Perinstance algorithm selection also shows much promise for boosting performance in solving continuous and mixed discrete/continuous optimisation problems.This survey provides an overview of research in automated algorithm selection, ranging from early and seminal works to recent and promising application areas. Different from earlier work, it covers applications to discrete and continuous problems, and discusses algorithm selection in context with conceptually related approaches, such as algorithm configuration, scheduling or portfolio selection. Since informative and cheaply computable problem instance features provide the basis for effective perinstance algorithm selection systems, we also provide an overview of such features for discrete and continuous problems. Finally, we provide perspectives on future work in the area and discuss a number of open research challenges.

show abstract

“…The results indicated that ensemble method was more effective for diagnosing valvular heart disease. Rijn (19) build online performance estimation framework for dynamic data stream that weight the votes of individual classifiers members across the data stream and rely only on Hoeffding trees as base-level classifier. The performance was estimated using two functions based on window and fading factors.…”

Section: Related Workmentioning

confidence: 99%

Voting-Boosting: A novel machine learning ensemble for the prediction of Infants' Data

2020

IJST

View full text Add to dashboard Cite

Background/Objectives: Owing to the continuous increase of electronic records and recent advances in machine learning, various automated disease diagnosis tools have been developed and proposed in healthcare sector. In the present study, an ensemble methodology using voting and boosting techniques has been proposed for optimal selection of features and prediction of infants' data of India. Methods/Analysis: For feature selection, the best-first search algorithm of wrapper technique has been used in addition to votingboosting. The proposed ensemble consists of combination of heterogeneous classifiers including Random Forest, J48, JRip, CART and Stochastic Gradient Descent (SGD). The effectiveness of the proposed ensemble and single classifiers have been investigated in terms of classification accuracy, precision, f-measure, recall, MCC and PRC area using varied k-fold cross validation. Findings: The results depicted that the proposed Voting-Boosting ensemble (k=15) outperforms the individual classifiers using selected features. Applications / Improvements: The proposed Voting-Boosting ensemble can be extended by using more state-of-the art classification approaches and further utilized for other healthcare datasets for enhancing the performance.

show abstract

The online performance estimation framework: heterogeneous ensemble learning for data streams

Cited by 101 publications

References 43 publications

Arbitrage of forecasting experts

Arbitrage of forecasting experts

Automated Algorithm Selection: Survey and Perspectives

Voting-Boosting: A novel machine learning ensemble for the prediction of Infants' Data

Contact Info

Product

Resources

About