Probabilistic Active Learning in Datastreams

Kottke, Daniel; Krempl, Georg; Spiliopoulou, Myra

doi:10.1007/978-3-319-24465-5_13

Cited by 16 publications

(20 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, if the stream is infinite but the budget is fixed, the AL process will stop at some point in time when the budget is exhausted, whereas the stream will continue indefinitely, rendering the adaptation of the ML classifier to structural changes in the stream difficult. In Kottke et al (2015), such a budget is considered and the use of an adaptive threshold is proposed on some computed quantity, which ensures that the Oracle budget is met within some tolerance window.…”

Section: Budgetmentioning

confidence: 99%

Coupling digital simulation and machine learning metamodel through an active learning approach in Industry 4.0 context

Chabanet

El-Haouzi

Thomas

2021

Computers in Industry

View full text Add to dashboard Cite

Although digital simulations are becoming increasingly important in the industrial world owing to the transition toward Industry 4.0, as well as the development of digital twin technologies, they have become increasingly computationally intensive. Many authors have proposed the use of Machine Learning (ML) metamodels to alleviate this cost and take advantage of the enormous amount of data that are currently available in industry. In an industrial context, it is necessary to continuously train predictive models integrated into decision support systems to ensure the consistency of their prediction quality over time. This led the authors to investigate Active Learning (AL) concepts in the particular context of the sawmilling industry. In this paper, a method based on AL is proposed to combine simulation and an ML metamodel that is trained incrementally using only selected data (smart data). A case study based on the sawmilling industry and experiments are shown, the results of which prove the possible advantages of this approach.

show abstract

Section: Budgetmentioning

confidence: 99%

Coupling digital simulation and machine learning metamodel through an active learning approach in Industry 4.0 context

Chabanet

El-Haouzi

Thomas

2021

Computers in Industry

View full text Add to dashboard Cite

show abstract

“…We combine the metric which we derived from [3] with a budget management strategy for traditional AL which was proposed by Kottke et al [2]. The budget strategy is based on an incremental percentile filter which keeps a sorted list of usefulness values and makes an acquisition if the incoming usefulness value of a new instance is in the top percentile of the sorted list.…”

Section: Related Workmentioning

confidence: 99%

“…In this section we describe two different budget management (BM) strategies. The first one was developed by Kottke et al [2] and takes the aforementioned instance quality into account. The second one on the other hand is a simple method that ignores the instance quality and was developed in order to investigate the effect of the budgeting on the AFA process.…”

Section: Budget Management With and Without Instance Qualitymentioning

confidence: 99%

“…The incremental percentile filter is a dynamic online budget manager as a means to quickly choose more qualitative instances while adapting to the changing usefulness a certain decision provides [2]. Its core idea is to store a limited amount of usefulness values in two windows: one sorted by the value itself, the other in the order they were received.…”

Section: Incremental Percentile Filter Budgetingmentioning

confidence: 99%

“…In this paper we are interested in a stream-based scenario where the decision if a feature should be acquired for an instance must be made immediately before seeing the next instance of the data stream. In order to achieve that goal we used an existing budgeting strategy from a stream-based AL algorithm that requires an instance quality estimate [2] and derived such estimates. These quality estimates are based on multiple metrics developed in [3], Average Euclidean Distance, Information Gain and Symmetric Uncertainty, all of which were formerly used in an active feature selection (AFS) scenario.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Active feature acquisition on data streams under feature drift

et al. 2020

Self Cite

View full text Add to dashboard Cite

Traditional active learning tries to identify instances for which the acquisition of the label increases model performance under budget constraints. Less research has been devoted to the task of actively acquiring feature values, whereupon both the instance and the feature must be selected intelligently and even less to a scenario where the instances arrive in a stream with feature drift. We propose an active feature acquisition strategy for data streams with feature drift, as well as an active feature acquisition evaluation framework. We also implement a baseline that chooses features randomly and compare the random approach against eight different methods in a scenario where we can acquire at most one feature at the time per instance and where all features are considered to cost the same. Our initial experiments on 9 different data sets, with 7 different degrees of missing features and 8 different budgets show that our developed methods outperform the random acquisition on 7 data sets and have a comparable performance on the remaining two.

show abstract

A Taxonomy of Interactive Online Machine Learning Strategies

Tegen

Davidsson

Persson

2021

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

In interactive machine learning, human users and learning algorithms work together in order to solve challenging learning problems, e.g. with limited or no annotated data or trust issues. As annotating data can be costly, it is important to minimize the amount of annotated data needed for training while still getting a high classification accuracy. This is done by attempting to select the most informative data instances for training, where the amount of instances is limited by a labelling budget. In an online learning setting, the decision of whether or not to select an instance for labelling has to be done on-the-fly, as the data arrives in a sequential order and is only valid for a limited time period. We present a taxonomy of interactive online machine learning strategies. An interactive learning strategy determines which instances to label in an unlabelled dataset. In the taxonomy we differentiate between interactive learning strategies when the computer controls the learning process (active learning) and those when human users control the learning process (machine teaching). We then make a distinction between what triggers the learning: active learning could be triggered by uncertainty, time, or randomly, whereas machine teaching could be triggered by errors, state changes, time, or factors related to the user. We also illustrate the taxonomy by implementing versions of the different strategies and performing experiments on a benchmark dataset as well as on a synthetically generated dataset. The results show that the choice of interactive learning strategy affects performance, especially in the beginning of the online learning process, when there is a limited amount of labelled data.

show abstract

Probabilistic Active Learning in Datastreams

Cited by 16 publications

References 18 publications

Coupling digital simulation and machine learning metamodel through an active learning approach in Industry 4.0 context

Coupling digital simulation and machine learning metamodel through an active learning approach in Industry 4.0 context

Active feature acquisition on data streams under feature drift

A Taxonomy of Interactive Online Machine Learning Strategies

Contact Info

Product

Resources

About