PT4AL: Using Self-supervised Pretext Tasks for Active Learning

sMoBYAL: supervised contrastive active learning for image classification

“…Active Learning methodologies aim to construct an efficiently labeled dataset. Recent approaches can generally be categorized into two types: uncertainty-based [4] [5], representative-based [6] [7], or a hybrid [8] of these two methods. Uncertainty-based methodologies target the identification of data samples that carry beneficial information for the model.…”

“…Hybrid methods employ a combination of the two above-mentioned philosophies to leverage their advantages and overcome their shortcomings. PT4AL [8] is a method of this type, where the author utilizes the correlation between the pretext task and the main task to group data into clusters, combined with the use of uncertainty to query data samples. However, a common issue that arises is the trade-off between the two factors of informativeness and diversity.…”

“…The core idea behind MoCo is the utilization of a momentum encoder. BYOL [11], on the other hand, eliminates the need for negative samples, which helps simplify the process of contrastive learning. BYOL employs a target encoder and an online encoder.…”

“…We designed a contrastive active learning framework based on MoBY [14]. This method combines two very successful methods in self-supervised learning, MoCo [10] and BYOL [11]. It reuses momentum design and memory bank from MoCo and inherits asymmetric data augmentations from BYOL.…”

“…This projection module is very important in improving the representation quality for contrastive loss. Unlike the target branch, the online branch also has a predictor module q that helps to create asymmetric architecture between the online branch and the target branch [11]. At the target branch, we also have a memory bank M 𝑈 to store the representation after being normalized.…”