CluHTM - Semantic Hierarchical Topic Modeling based on CluWords

Abstract

Hierarchical Topic modeling (HTM) exploits latent topics and relationships among them as a powerful tool for data analysis and exploration. Despite advantages over traditional topic modeling, HTM poses its own challenges, such as (1) topic incoherence, (2) unreasonable (hierarchical) structure, and(3) issues related to the definition of the "ideal" number of topics and depth of the hierarchy. In this paper, we advance the stateof-the-art on HTM by means of the design and evaluation of CluHTM, a novel nonprobabilistic hierarchical matrix factorization aimed at solving the specific issues of HTM. CluHTM's novel contributions include: (i) the exploration of richer text representation that encapsulates both, global (dataset level) and local semantic information -when combined, these pieces of information help to solve the topic incoherence problem as well as issues related to the unreasonable structure; (ii) the exploitation of a stability analysis metric for defining the number of topics and the "shape" the hierarchical structure. In our evaluation, considering twelve datasets and seven stateof-the-art baselines, CluHTM outperformed the baselines in the vast majority of the cases, with gains of around 500% over the strongest state-of-the-art baselines. We also provide qualitative and quantitative statistical analyses of why our solution works so well.

Tree-Structured Topic Modeling with Nonparametric Neural Variational Inference

“…As mentioned in (Viegas et al, 2020), a reasonable topic hierarchy means that topics near the root should be more general while the ones close to the leaves should be more specific. To this end, we adopt topic specialization (Kim et al, 2012) as an indicator for the evaluation of topical hierarchy.…”

“…The reason may be that each document is generated by topics along a single path for hLDA, which renders the large specialization of the topics at level 3 since they are all restricted to one topic from level 2. A reasonable topic hierarchy also indicates that child topics are coherent with their corresponding parent topics (Viegas et al, 2020). To measure the relations of two connected topics, we develop a new metric named cross-level NPMI (CLNPMI) to measure the relations of two connected topics by calculating the average NPMI value of every two different topic words from a parent topic and its child.…”

“…Liu et al (2018) used non-negative matrix factorization (NMF) with three optimization constraints, including global independence, local independence, and information consistency, to preserve topic coherence and a reasonable structure. Viegas et al (2020) incorporated pre-trained word embeddings into NMF to further improve topic coherence. The main limitation of NMF-based methods, however, is that a time-consuming process (e.g., measure the stability of results by running multiple random samplings) is necessary to determine the number of topics at each level.…”

“…Despite the practical importance and potential advantages, tree-structured topic models still face the following challenges. Firstly, the hierarchical structure of topics should be reasonable (Viegas et al, 2020). Typically, topics near the root are more general while the ones close to the leaves are more specific.…”

TAN-NTM: Topic Attention Networks for Neural Topic Modeling

“…Hierarchical topic models (Viegas et al, 2020) utilize relationships among the latent topics. Supervised topic models have been explored previously where the topic model is trained through human feedback (Kumar et al, 2019) or with a task specific network simultaneously such that topic extraction is guided through task labels (Pergola et al, 2019;Wang and Yang, 2020).…”