Pre-Training Acquisition Functions by Deep Reinforcement Learning for Fixed Budget Active Learning

Abstract

There are many situations in supervised learning where the acquisition of data is very expensive and sometimes determined by a user’s budget. One way to address this limitation is active learning. In this study, we focus on a fixed budget regime and propose a novel active learning algorithm for the pool-based active learning problem. The proposed method performs active learning with a pre-trained acquisition function so that the maximum performance can be achieved when the number of data that can be acquired is fixed. To implement this active learning algorithm, the proposed method uses reinforcement learning based on deep neural networks as as a pre-trained acquisition function tailored for the fixed budget situation. By using the pre-trained deep Q-learning-based acquisition function, we can realize the active learner which selects a sample for annotation from the pool of unlabeled samples taking the fixed-budget situation into account. The proposed method is experimentally shown to be comparable with or superior to existing active learning methods, suggesting the effectiveness of the proposed approach for the fixed-budget active learning.

Stopping Criterion for Active Learning Based on Error Stability

“…These two approaches can be combined to achieve the optimal acquisition function (Xu et al, 2003;Donmez et al, 2007;Huang et al, 2010;Karzand and Nowak, 2020). Recently, as another approach, methods of learning acquisition functions have also been proposed (Konyushkova et al, 2017;Sener and Savarese, 2018;Taguchi et al, 2021).…”