Distribution theory for hierarchical processes

Abstract

Hierarchies of discrete probability measures are remarkably popular as nonparametric priors in applications, arguably due to two key properties: (i) they naturally represent multiple heterogeneous populations; (ii) they produce ties across populations, resulting in a shrinkage property often described as "sharing of information". In this paper we establish a distribution theory for hierarchical random measures that are generated via normalization, thus encompassing both the hierarchical Dirichlet and hierarchical Pitman-Yor processes. These results provide a probabilistic characterization of the induced (partially exchangeable) partition structure, including the distribution and the asymptotics of the number of partition sets, and a complete posterior characterization. They are obtained by representing hierarchical processes in terms of completely random measures, and by applying a novel technique for deriving the associated distributions. Moreover, they also serve as building blocks for new simulation algorithms, and we derive marginal and conditional algorithms for Bayesian inference.

Hierarchical Species Sampling Models

“…The results in Proposition 6 generalize to HSSM those given in Theorem 5 ofCamerlenghi et al (2018) for HNRMI. Our proof relies on the hierarchical SSS construction (see Proposition 4), whereas the proof inCamerlenghi et al (2018) builds on the partial exchangeable partition function given in Proposition 5.…”

“…By exploiting the properties of hierarchical species sampling sequences, we are able to provide the finite sample distribution of the number of clusters for each group of observations and the total number of clusters. Moreover, we provide some new asymptotic results when the number of observations goes to infinity, thus extending the asymptotic approximations for species sampling given in Pitman (2006)) and for hierarchical normalized random measures given in Camerlenghi et al (2018).…”

“…Remark 4. Part (ii) extends to HSSM with different group sizes, n i (t), the results given in Theorem 7 of Camerlenghi et al (2018) for HNRMI with groups of equal size. Both part (i) and (ii) provide deterministic scaling of diversities, in the spirit of Pitman (2006), and differently from Camerlenghi et al (2018) where a random scaling is obtained.…”

“…Remark 3. For a suitable choice of q and q 0 , one obtains as special cases of Proposition 5 the results in Theorem 3 and 4 of Camerlenghi et al (2018). The proof of Proposition 5 relies on combinatorical arguments combined with the results in Proposition 3 and 4, whereas Camerlenghi et al (2018) use mainly specific properties of normalized random measures.…”

“…The proposed framework is general enough to provide a probabilistic foundation of both existing and novel hierarchical random measures, also allowing for non diffuse base measures. Our HSSM class includes the HDP, its generalization given by the Hierarchical Pitman-Yor process (HPYP), see Teh (2006); Du et al (2010); Lim et al (2016) and the hierarchical normalized random measures with independent increments (HNRMI), recently studied in Camerlenghi et al (2018). Among the novel measures, we study the hierarchical generalization of the Gnedin process (Gnedin (2010)) and of finite mixtures (e.g., Miller and Harrison (2017)) and the asymmetric hierarchical constructions with p 0 and p i of different type (Du et al (2010); Buntine and Mishra (2014)).…”