Optimised probabilistic active learning (OPAL)

Krempl, Georg; Kottke, Daniel; Lemaire, Vincent

doi:10.1007/s10994-015-5504-1

Cited by 37 publications

(16 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…at the beginning of the training) and therefore suggested the use of a beta prior. Krempl et al (2015) and Kottke et al (2016) address the issue pointed out by Chapelle and named their approach probabilistic AL. They propose to use a distribution of the class posterior probability instead of using the classifier outputs directly.…”

Section: Introductionmentioning

confidence: 99%

Toward optimal probabilistic active learning using a Bayesian approach

et al. 2021

Self Cite

View full text Add to dashboard Cite

Gathering labeled data to train well-performing machine learning models is one of the critical challenges in many applications. Active learning aims at reducing the labeling costs by an efficient and effective allocation of costly labeling resources. In this article, we propose a decision-theoretic selection strategy that (1) directly optimizes the gain in misclassification error, and (2) uses a Bayesian approach by introducing a conjugate prior distribution to determine the class posterior to deal with uncertainties. By reformulating existing selection strategies within our proposed model, we can explain which aspects are not covered in current state-of-the-art and why this leads to the superior performance of our approach. Extensive experiments on a large variety of datasets and different kernels validate our claims.

show abstract

Section: Introductionmentioning

confidence: 99%

Toward optimal probabilistic active learning using a Bayesian approach

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Overall, a small improvement in a dense region may thus be more beneficial than a big improvement in a sparse region. This observation motivates a kind of density weighting, i.e., the combination (multiplication) of an uncertainty degree with the (estimated) density of a data point (Krempl et al 2015). • Last but not least, going beyond uncertainty sampling for binary classification as considered in this paper, the idea of epistemic uncertainty sampling should also be extended toward other learning problems, such as multi-class classification and regression.…”

Section: Discussionmentioning

confidence: 99%

How to measure uncertainty in uncertainty sampling for active learning

2021

View full text Add to dashboard Cite

Various strategies for active learning have been proposed in the machine learning literature. In uncertainty sampling, which is among the most popular approaches, the active learner sequentially queries the label of those instances for which its current prediction is maximally uncertain. The predictions as well as the measures used to quantify the degree of uncertainty, such as entropy, are traditionally of a probabilistic nature. Yet, alternative approaches to capturing uncertainty in machine learning, alongside with corresponding uncertainty measures, have been proposed in recent years. In particular, some of these measures seek to distinguish different sources and to separate different types of uncertainty, such as the reducible (epistemic) and the irreducible (aleatoric) part of the total uncertainty in a prediction. The goal of this paper is to elaborate on the usefulness of such measures for uncertainty sampling, and to compare their performance in active learning. To this end, we instantiate uncertainty sampling with different measures, analyze the properties of the sampling strategies thus obtained, and compare them in an experimental study.

show abstract

“…ADASYN [13] uses a weighted distribution for different minority class according to their level of difficulty in learning and more synthetic data for minority class. A typical algorithm level approach is called cost-sensitive learning [18], [19], [20], [21], [22]. We focus the study of algorithm level approach in this paper.…”

Section: Introductionmentioning

confidence: 99%

A Cost-Sensitive Deep Belief Network for Imbalanced Classification

Zhang

Tan

et al. 2019

IEEE Trans. Neural Netw. Learning Syst.

226

View full text Add to dashboard Cite

Imbalanced data with a skewed class distribution are common in many real-world applications. Deep Belief Network (DBN) is a machine learning technique that is effective in classification tasks. However, conventional DBN does not work well for imbalanced data classification because it assumes equal costs for each class. To deal with this problem, cost-sensitive approaches assign different misclassification costs for different classes without disrupting the true data sample distributions. However, due to lack of prior knowledge, the misclassification costs are usually unknown and hard to choose in practice. Moreover, it has not been well studied as to how cost-sensitive learning could improve DBN performance on imbalanced data problems. This paper proposes an evolutionary cost-sensitive deep belief network (ECS-DBN) for imbalanced classification. ECS-DBN uses adaptive differential evolution to optimize the misclassification costs based on the training data that presents an effective approach to incorporating the evaluation measure (i.e., G-mean) into the objective function. We first optimize the misclassification costs, and then apply them to DBN. Adaptive differential evolution optimization is implemented as the optimization algorithm that automatically updates its corresponding parameters without the need of prior domain knowledge. The experiments have shown that the proposed approach consistently outperforms the state of the art on both benchmark data sets and real-world data set for fault diagnosis in tool condition monitoring.

show abstract

Optimised probabilistic active learning (OPAL)

Cited by 37 publications

References 21 publications

Toward optimal probabilistic active learning using a Bayesian approach

Toward optimal probabilistic active learning using a Bayesian approach

How to measure uncertainty in uncertainty sampling for active learning

A Cost-Sensitive Deep Belief Network for Imbalanced Classification

Contact Info

Product

Resources

About