Multi-Armed Bandit for Species Discovery: A Bayesian Nonparametric Approach

Abstract

ABSTRACT. Let (P 1 , . . . , P J ) denote J populations of animals from distinct regions.A priori, it is unknown which species are present in each region and what are their corresponding frequencies. Species are shared among populations and each species can be present in more than one region with its frequency varying across populations. In this paper we consider the problem of sequentially sampling these populations in order to observe the greatest number of di↵erent species. We adopt a Bayesian nonparametric approach and endow (P 1 , . . . , P J ) with a Hierarchical Pitman-Yor process prior. As a consequence of the hierarchical structure, the J unknown discrete probability measures share the same support, that of their common random base measure. Given this prior choice, we propose a sequential rule that, at every time step, given the information available up to that point, selects the population from which to collect the next observation. Rather than picking the population with the highest posterior estimate of producing a new value, the proposed rule includes a Thompson sampling step to 1 better balance the exploration-exploitation trade-o↵. We also propose an extension of the algorithm to deal with incidence data, where multiple observations are collected in a time period. The performance of the proposed algorithms is assessed through a simulation study and compared to three other strategies. Finally, we compare these algorithms using a dataset of species of trees, collected from di↵erent plots in South America.

Microbial diversity estimation and hill number calculation using the hierarchical Pitman-Yor process

“…Buntine and Hutter (2010) developed a sampler in this style which could be applied instead of the table indicator sampler as described by Battiston et al (2018). Similarly, there have also been developments in variational approximations in Dirichlet mixtures that could be extended for the hierarchical Pitman-Yor model (Huynh et al, 2016;Kurihara et al, 2007).…”

“…The species of each individual in the population is determined by the sample. Therefore, when considering the full conditional distribution of each t jp , we must also condition on the observed species of individual p. Here we use the method presented in Battiston et al (2018) wherein each ancestral state is updated by first "removing" individual p from its population and either reallocating it to an existing ancestor (of the same species) or allocating a new ancestor.…”

Perfect Sampling of the Posterior in the Hierarchical Pitman–Yor Process

“…See Teh and Jordan (2010) for a review on hierarchical nonparametric priors. In Bayesian nonparametrics, theoretical developments and applications of the hierarchical Pitman-Yor process have been considered in language modeling (Teh, 2006;Huang and Renals, 2007;Wood et al, 2009), infinite hidden Markov modeling (Beal et al, 2002;Van Gael et al, 2008;Blunsom and Cohn, 2011), species sampling with multiple populations (Battiston et al, 2018;Bassetti et al, 2020;, clustering (Argiento et al, 2020), graphical modeling (Creamschi et al, 2020), image segmentation (Sudderth and Jordan, 2009), and topic models (Sato and Nakagawa, 2010;Araki et al, 2012;Lindsey et al, 2012). In this paper we evaluate and compare multiple computational strategies for posterior inference under the hierarchical Pitman-Yor process prior.…”

Nonparametric Bayesian multi-armed bandits for single cell experiment design

“…Recent work [Bubeck et al, 2013, Battiston et al, 2018, Dumitrascu et al, 2018b proposed the use of classical multi-armed bandit strategies, e.g., upper confidence bounds (UCB) Robbins, 1985, Auer et al, 2002] and Thompson sampling (TS) [Thompson, 1933], for devising sequential approaches to maximize the number of distinct species discovered by sampling over multiple populations. These sampling strategies balance the exploration of the experimental choices-which populations are sampled-with the exploitation of populations that maximize current estimates of the expected rewards-the observed species diversity within a population.…”