ALLSH: Active Learning Guided by Local Sensitivity and Hardness

Abstract

Active learning, which effectively collects informative unlabeled data for annotation, reduces the demand for labeled data. In this work, we propose to retrieve unlabeled samples with a local sensitivity and hardnessaware acquisition function. The proposed method generates data copies through local perturbations and selects data points whose predictive likelihoods diverge the most from their copies. We further empower our acquisition function by injecting the select-worst case perturbation. Our method achieves consistent gains over the commonly used active learning strategies in various classification tasks. Furthermore, we observe consistent improvements over the baselines on the study of prompt selection in prompt-based few-shot learning. These experiments demonstrate that our acquisition guided by local sensitivity and hardness can be effective and beneficial for many NLP tasks.

A Unified Framework for Alternating Offline Model Training and Policy Learning

“…Rather than using a fixed MLE-trained model, we derive an objective that trains both the policy and the dynamic model toward maximizing a lower bound of true expected return (simultaneously minimizing the policy evaluation error |J(π, P * ) − J(π, P )|). Several recent works also propose to enhance model training, such as training a reverse dynamic model to encourage conservative model-based imaginations [53], learning a balanced state-action representation for the model to mitigate the distributional discrepancy [54,55], and using advanced dynamic-model architecture of the GPT-Transformer [56][57][58][59] to achieve accurate predictions [60]. These methods are orthogonal to ours and may further improve the performance.…”