An Active Learning Approach with Uncertainty, Representativeness, and Diversity

Abstract

Big data from the Internet of Things may create big challenge for data classification. Most active learning approaches select either uncertain or representative unlabeled instances to query their labels. Although several active learning algorithms have been proposed to combine the two criteria for query selection, they are usually ad hoc in finding unlabeled instances that are both informative and representative and fail to take the diversity of instances into account. We address this challenge by presenting a new active learning framework which considers uncertainty, representativeness, and diversity creation. The proposed approach provides a systematic way for measuring and combining the uncertainty, representativeness, and diversity of an instance. Firstly, use instances' uncertainty and representativeness to constitute the most informative set. Then, use the kernel k-means clustering algorithm to filter the redundant samples and the resulting samples are queried for labels. Extensive experimental results show that the proposed approach outperforms several state-of-the-art active learning approaches.

Active Learning for BERT: An Empirical Study

“…In contrast, the other strategies tend to select less diverse batches, i.e., they are prone to choose redundant examples, especially in the imbalanced-practical scenario. Thus, combining these approaches with methods that encourage diversity (e.g., He et al, 2014;Zhdanov, 2019;Ash et al, 2019) can potentially lead to further improvement in their resultant prediction performance. In terms of representativeness, DAL, which is a representativeness-driven method, again consistently leads across the scenarios.…”

Pool-Based Sequential Active Learning for Regression

“…Both strategies are specific to the support vector machine classifier. He et al [18] considered uncertainty, representativeness, information content, and diversity in batch-mode ALC. Let k be the batch size.…”

Practical, Efficient, and Customizable Active Learning for Named Entity Recognition in the Digital Humanities

“…Shen et al (2004) define three broad criteria for determining which data will be most informative to the model if annotated: uncertainty, where instances which confuse the model are given priority; diversity, where instances that would expand the model's coverage are prioritized; and representativeness, prioritizing instances that best approximate the true distribution over all instances. Uncertainty-based approaches outperform other single-criterion approaches, though many works, primarily in Computer Vision, demonstrate that considering diversity reduces repetitive training examples and representativeness reduces outlier sampling (Roy and McCallum, 2001;Zhu et al, 2003;Settles and Craven, 2008;Zhu et al, 2008;Olsson, 2009;Gu et al, 2014;He et al, 2014;Yang et al, 2015;Wang et al, 2018b).…”