Task-Driven Comparison of Topic Models

Abstract

Topic modeling, a method of statistically extracting thematic content from a large collection of texts, is used for a wide variety of tasks within text analysis. Though there are a growing number of tools and techniques for exploring single models, comparisons between models are generally reduced to a small set of numerical metrics. These metrics may or may not reflect a model's performance on the analyst's intended task, and can therefore be insufficient to diagnose what causes differences between models. In this paper, we explore task-centric topic model comparison, considering how we can both provide detail for a more nuanced understanding of differences and address the wealth of tasks for which topic models are used. We derive comparison tasks from single-model uses of topic models, which predominantly fall into the categories of understanding topics, understanding similarity, and understanding change. Finally, we provide several visualization techniques that facilitate these tasks, including buddy plots, which combine color and position encodings to allow analysts to readily view changes in document similarity.

Visual exploration of topic variants through a hybrid biclustering approach

“…En effet, aussi bien les termes que les documents peuvent concerner plusieurs sujets. D'autre part, les approches probabilistes produisant les partitions non-disjointes sont difficiles à appréhender par un public non averti [2]. Nous avons déjà proposé quelques solutions dans la carte pondérée des sujets en affichant les relations de similarité entre les sujets, aidant par ailleurs les journalistes dans leur processus de diversification.…”

SolarView: Low Distortion Radial Embedding with a Focus

“…A related approach for displaying distances between a focus and its close neighbors is the buddy plot from Alexander and Gleicher [17]. In a buddy plot, the focus is displayed to the left of a line segment, and its neighbors are displayed as discs on that line.…”

Recent research advances on interactive machine learning

“…Along the same line, Thom et al (Thom et al, 2015) juxtapose the word clouds that visualize the keywords of topics, to enable a semantic comparison between topics for situational awareness analysis on social media. The between-model topic comparison can help validate the gained insights (Alexander and Gleicher, 2016), select models and tune their parameters (Alexander and Gleicher, 2016;El-Assady et al, 2018). Alexander and Gleicher (Alexander and Gleicher, 2016) propose a task-driven approach for between-model topic comparison.…”

“…The between-model topic comparison can help validate the gained insights (Alexander and Gleicher, 2016), select models and tune their parameters (Alexander and Gleicher, 2016;El-Assady et al, 2018). Alexander and Gleicher (Alexander and Gleicher, 2016) propose a task-driven approach for between-model topic comparison. They characterized three main specific comparison tasks: topic alignment, distance comparison and timeline comparison.…”