Active Learning Is Planning: Nonmyopic ε-Bayes-Optimal Active Learning of Gaussian Processes

Abstract

Abstract. A fundamental issue in active learning of Gaussian processes is that of the exploration-exploitation trade-off. This paper presents a novel nonmyopic -Bayes-optimal active learning ( -BAL) approach [4] that jointly optimizes the trade-off. In contrast, existing works have primarily developed greedy algorithms or performed exploration and exploitation separately. To perform active learning in real time, we then propose an anytime algorithm [4] based on -BAL with performance guarantee and empirically demonstrate using a real-world dataset that, with limited budget, it outperforms the state-of-the-art algorithms.

Recent Advances in Scaling Up Gaussian Process Predictive Models for Large Spatiotemporal Data

“…To address this trade-off, one principled approach is to frame active sensing as a sequential decision problem that jointly optimizes the above exploration-exploitation trade-off while maintaining a Bayesian belief over the model parameters. Solving this problem then results in an induced policy that is guaranteed to be optimal in the expected active sensing performance [13]. Unfortunately, such a nonmyopic Bayes-optimal active learning (BAL) policy cannot be derived exactly due to an uncountable set of candidate observations and unknown model parameters.…”

“…As a result, existing works advocate using greedy policies [24] or performing exploration and exploitation separately [15] to sidestep the difficulty of solving for the exact BAL policy. But, these algorithms are sub-optimal in the presence of budget constraints due to their imbalance between exploration and exploitation [13].…”

“…Our final work proposes a novel nonmyopic active sensing/learning algorithm [13,12] (Section 5) that can still preserve and exploit the principled Bayesian sequential decision problem framework for jointly optimizing the exploration-exploitation trade-off and hence does not incur the limitations of existing works. In particular, although the exact BAL policy cannot be derived, we show that it is in fact possible to solve for a nonmyopic -Bayes-optimal active learning ( -BAL) policy given an arbitrary loss bound .…”

“…To cast active sensing as a Bayesian sequential decision problem, we define a sequential active sensing policy π {π n } N n=1 that is structured to sequentially decide the next location π n (z D ) ∈ X \ D to be observed at each stage n based on the current observations z D over a finite planning horizon of N stages (i.e., sampling budget). To measure the predictive uncertainty over unobserved areas of the phenomenon, we use the entropy criterion and define the value under a policy π to be the joint entropy of its selected observations when starting with some prior observations z D0 and following π thereafter [13]. The work of [19] has established that minimizing the posterior joint entropy (i.e., predictive uncertainty) remaining in unobserved locations of the phenomenon is equivalent to maximizing the joint entropy of π.…”

“…The work of [19] has established that minimizing the posterior joint entropy (i.e., predictive uncertainty) remaining in unobserved locations of the phenomenon is equivalent to maximizing the joint entropy of π. Thus, solving the active sensing problem entails choosing a sequential BAL policy π * n (z D ) = arg max x∈X \D Q * n (z D , x) induced from the following N -stage Bellman equations, as formally derived in [13]:…”