Region-Based Active Learning with Hierarchical and Adaptive Region Construction

Luo, Zhipeng; Hauskrecht, Miloš

doi:10.1137/1.9781611975673.50

Cited by 6 publications

(2 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Region queries do not query information regarding a specific instance, but ask annotators to provide information about an entire region in the feature space [102]. For this purpose, the query is to be formulated in an appropriate and human-readable representation [103]. A common way to achieve this requirement involves formulating premises of sharp or possibilistic classification rules by defining conditions on the value ranges of features [104].…”

Section: A Query Typesmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Cost Types, Interaction Schemes, and Annotator Performance Models in Selection Algorithms for Active Learning in Classification

et al. 2021

View full text Add to dashboard Cite

Pool-based active learning (AL) aims to optimize the annotation process (i.e., labeling) as the acquisition of annotations is often time-consuming and therefore expensive. For this purpose, an AL strategy queries annotations intelligently from annotators to train a high-performance classification model at a low annotation cost. Traditional AL strategies operate in an idealized framework. They assume a single, omniscient annotator who never gets tired and charges uniformly regardless of query difficulty. However, in real-world applications, we often face human annotators, e.g., crowd or in-house workers, who make annotation mistakes and can be reluctant to respond if tired or faced with complex queries. Recently, a wide range of novel AL strategies has been proposed to address these issues. They differ in at least one of the following three central aspects from traditional AL: (1) They explicitly consider (multiple) human annotators whose performances can be affected by various factors, such as missing expertise. (2) They generalize the interaction with human annotators by considering different query and annotation types, such as asking an annotator for feedback on an inferred classification rule. (3) They take more complex cost schemes regarding annotations and misclassifications into account. This survey provides an overview of these AL strategies and refers to them as real-world AL. Therefore, we introduce a general real-world AL strategy as part of a learning cycle and use its elements, e.g., the query and annotator selection algorithm, to categorize about 60 real-world AL strategies. Finally, we outline possible directions for future research in the field of AL.

show abstract

Section: A Query Typesmentioning

confidence: 99%

“…indicating the proportion of positive instances Du and Ling [10], Luo and Hauskrecht [117,118,103], Rashidi and Cook [104], Haque et al [119] What is the proportion of positive instances in the region described by the feature constellation, e.g.,…”

Section: Strategy Query Annotationmentioning

confidence: 99%