Nested Hierarchical Dirichlet Process for Nonparametric Entity-Topic Analysis

Abstract

Abstract. The Hierarchical Dirichlet Process (HDP) is a Bayesian nonparametric prior for grouped data, such as collections of documents, where each group is a mixture of a set of shared mixture densities, or topics, where the number of topics is not fixed, but grows with data size. The Nested Dirichlet Process (NDP) builds on the HDP to cluster the documents, but allowing them to choose only from a set of specific topic mixtures. In many applications, such a set of topic mixtures may be identified with the set of entities for the collection. However, in many applications, multiple entities are associated with documents, and often the set of entities may also not be known completely in advance. In this paper, we address this problem using a nested HDP (nHDP), where the base distribution of an outer HDP is itself an HDP. The inner HDP creates a countably infinite set of topic mixtures and associates them with entities, while the outer HDP associates documents with these entities or topic mixtures. Making use of a nested Chinese Restaurant Franchise (nCRF) representation for the nested HDP, we propose a collapsed Gibbs sampling based inference algorithm for the model. Because of couplings between two HDP levels, scaling up is naturally a challenge for the inference algorithm. We propose an inference algorithm by extending the direct sampling scheme of the HDP to two levels. In our experiments on two real world research corpora, we show that, even when large fractions of author entities are hidden, the nHDP is able to generalize significantly better than existing models. More importantly, we are able to detect missing authors at a reasonable level of accuracy.

Flexible clustering via hidden hierarchical Dirichlet priors

“…Importantly, the HHDP overcomes the above-mentioned theoretical, modeling, and computational limitations since it, respectively, does not suffer from the degeneracy flaw, is able to effectively capture different weights of shared clusters and allows to handle several populations as showcased in the real data application. Note that the idea of the model was first hinted at in James (2008) and, later, considered in Agrawal et al (2013) from a mere computational point of view without providing results on distributional properties that are relevant for Bayesian inference. Hence, as a by-product, our theoretical results shed also some light on the topic modeling applications of Agrawal et al (2013).…”

“…Note that the idea of the model was first hinted at in James (2008) and, later, considered in Agrawal et al (2013) from a mere computational point of view without providing results on distributional properties that are relevant for Bayesian inference. Hence, as a by-product, our theoretical results shed also some light on the topic modeling applications of Agrawal et al (2013). Additionally, the same model was independently applied in Balocchi et al (2021) to successfully cluster urban areal units at different levels of resolution simultaneously.…”

Clustering Areal Units at Multiple Levels of Resolution to Model Crime in Philadelphia

“…The models developed by Denti et al (2021) and Beraha et al (2021) similarly address the flexibility constraint in the nDP and combine the nDP with models suitable for grouped data, but different from the HDP. Agrawal et al (2013), Wulsin et al (2016) and Nguyen et al (2014) use a model that is essentially equal to the one used in this work, but apply it in different contexts; moreover Nguyen et al (2014) consider the particular case where additional group-level data is available. Additionally, the same model was independently studied in Lijoi et al (2020), although their focus on the theoretical aspects of the model.…”

Contextual Face Recognition with a Nested-Hierarchical Nonparametric Identity Model

“…This is related to the dual-HDP described inWang et al (2009) and the single-entity model ofAgrawal et al (2013), for example, although these works tended to focus on textual topic modelling.…”