Is Your Anchor Going Up or Down? Fast and Accurate Supervised Topic Models

Abstract

Topic models provide insights into document collections, and their supervised extensions also capture associated document-level metadata such as sentiment. However, inferring such models from data is often slow and cannot scale to big data. We build upon the "anchor" method for learning topic models to capture the relationship between metadata and latent topics by extending the vector-space representation of word-cooccurrence to include metadataspecific dimensions. These additional dimensions reveal new anchor words that reflect specific combinations of metadata and topic. We show that these new latent representations predict sentiment as accurately as supervised topic models, and we find these representations more quickly without sacrificing interpretability.

Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledge

“…Using an assumption of separability, these anchor words act as high precision markers of particular topics and, thus, help discern the topics from one another. Although the original algorithm proposed by Arora et al (2012), and subsequent improvements to their approach, find these anchor words automatically (Arora et al, 2013;Lee and Mimno, 2014), recent adaptations allow manual insertion of anchor words and other metadata Nguyen et al, 2015). Our work is similar to the latter, where we treat anchor words as fuzzy logic markers and embed them into the topic model in a semi-supervised fashion.…”

Improving Topic Coherence Using Entity Extraction Denoising

“…As the acceptance of topic coherence measures increases as a mean of topic model assessment (Paul and Girju, 2010;Reisinger et al, 2010;Hall et al, 2012), recent research trends focus on proposing fast and efficient models that can be scaled up to big amounts of data (Yang et al, 2015;Nguyen et al, 2015), using the whole text per document for training.…”

Tea Party in the House: A Hierarchical Ideal Point Topic Model and Its Application to Republican Legislators in the 112th Congress

“…We also associate each second-level frame node with an ideal point η k,j ∼ N (0, γ). This resembles how supervised topic models (Blei and McAuliffe, 2007;Nguyen et al, 2015) discover polarized topics' associated response variables.…”