Relational Biterm Topic Model: Short-Text Topic Modeling using Word Embeddings

Li, Ximing; Zhang, Ang; Li, Changchun; Guo, Lanhui; Wang, Wenting; Ouyang, Jian

doi:10.1093/comjnl/bxy037

Cited by 34 publications

(18 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The application of topic models builds on the identification of coherently recurring word patterns (topics) within documents (Blei, 2012); short documents, however, often feature an insufficient frequency of word co-occurrences so that the topic modeling algorithm is unable to detect coherent patterns. This so-called "sparsity problem" leads to low quality topic models (Hong & Davison, 2010;Li et al, 2019). While a growing branch of research is concerned with developing new specialized models for collections of short texts (see e.g., Li et al, 2019), 9 a simple and established approach to address the sparsity-problem is to concatenate several short documents to a longer one (Guo et al, 2016).…”

Section: Topic Modelingmentioning

confidence: 99%

“…This so-called "sparsity problem" leads to low quality topic models (Hong & Davison, 2010;Li et al, 2019). While a growing branch of research is concerned with developing new specialized models for collections of short texts (see e.g., Li et al, 2019), 9 a simple and established approach to address the sparsity-problem is to concatenate several short documents to a longer one (Guo et al, 2016). While this artificial document concatenation distorts the structure of the original data, the method leads to a significant increase of topic coherence in the resulting models (Steinskog et al, 2017).…”

Section: Topic Modelingmentioning

confidence: 99%

See 1 more Smart Citation

Machine Translation Vs. Multilingual Dictionaries Assessing Two Strategies for the Topic Modeling of Multilingual Text Collections

Maier

Baden

Stoltenberg

et al. 2021

Communication Methods and Measures

View full text Add to dashboard Cite

The goal of this paper is to evaluate two methods for the topic modeling of multilingual document collections: (1) machine translation (MT), and (2) the coding of semantic concepts using a multilingual dictionary (MD) prior to topic modeling. We empirically assess the consequences of these approaches based on both a quantitative comparison of models and a qualitative validation of each method's potentials and weaknesses. Our case study uses two text collections (of tweets and news articles) in three languages (English, Hebrew, Arabic), covering the ongoing local conflicts between Israeli authorities, settlers, and Palestinian Bedouins in the West Bank. We find that both methods produce a large share of equivalent topics, especially in the context of fairly homogenous news discourse, yet show limited but systematic differences when applied to highly heterogenous social media discourse. While the MD model delivers a more nuanced picture of conflict-related topics, it misses several more peripheral topics, especially those unrelated to the dictionary's focus, which are picked up by the MT model. Our study is a first step toward instrument validation, indicating that both methods yield valid, comparable results, while method-specific differences remain.

show abstract

Section: Topic Modelingmentioning

confidence: 99%

Section: Topic Modelingmentioning

confidence: 99%

Machine Translation Vs. Multilingual Dictionaries Assessing Two Strategies for the Topic Modeling of Multilingual Text Collections

Maier

Baden

Stoltenberg

et al. 2021

Communication Methods and Measures

View full text Add to dashboard Cite

show abstract

“…This model tries to infer the topics in a corpus and the joint probability distribution for LDA can be factored as P( w,z ) = P( w|z )× P( z|r ). The success of LDA has motivated numerous models to extend it in order to analyse opinions (Blei, Ng, & Jordan, ; Koller & Friedman, ; Duan, Li, Li, Lu, & Wen, ; Chen, Jose, Yu, & Yuan, ; Li et al, ; Yang, Ma, Silva, Liu, & Hua, 2015; Yang et al, ; Li, Ouyang, & Zhou, ; Chien, Lee, & Tan, ).…”

Section: Related Workmentioning

confidence: 99%

Enriched Latent Dirichlet Allocation for Sentiment Analysis

2020

View full text Add to dashboard Cite

One of the main benefits of unsupervised learning is that there is no need for labelled data. As a method of this category, latent Dirichlet allocation (LDA) estimates the semantic relations between the words of the text effectively and can play an important role in solving various issues, including emotional analysis in combination with other parameters. In this study, three novel topic models called date sentiment LDA (DSLDA), author–date sentiment LDA (ADSLDA), and pack–author–date sentiment LDA (PADSLDA) are proposed. The proposed models extend LDA through some extra parameters such as date, author, helpfulness, sentiment, and subtopic. The proposed models use helpfulness in the Gibbs sampling algorithm. Helpfulness is a part of readers who found the review helpful. The proposed models divide the words into two categories: the words more affected by the distribution of subtopic and the words more affected by the main topic. In this study, a new concept called pack is introduced, and a new model called PADSLDA is proposed for sentiment analysis at pack level. The proposed models outperformed the baseline models because according to evaluations results, the extra parameters can appropriately affect the generating process of words in a review. Sentiment analysis at the document level, perplexity, and topic coherence are the main parameters used in the evaluations.

show abstract

“…In addition to the aforementioned aggregation or pooling based models, another popular research direction for short text topic modelling is using word correlations or embedding to enhance topic models. For example, Biterm Topic Model (BTM) (Yan et al, 2013) and Relational BTM (Li et al, 2018b) enrich each document with word pairs (i.e., biterms). Instead of using all the word pairs, Yang et al (2015a) considers pre-trained phrases.…”

Section: Related Workmentioning

confidence: 99%

Leveraging Meta Information in Short Text Aggregation

Zhao¹,

Du²,

Liu³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Analysing topics in short texts (e.g., tweets and new headings) is a challenging task because short texts often contain insufficient word co-occurrence information, which is important to learn good topics in conventional topic topics. To deal with the insufficiency, we propose a generative model that aggregates short texts into clusters by leveraging the associated meta information. Our model can generate more interpretable topics as well as document clusters. We develop an effective Gibbs sampling algorithm favoured by the fully local conjugacy in the model. Extensive experiments demonstrate that our model achieves better performance in terms of document clustering and topic coherence.

show abstract

Relational Biterm Topic Model: Short-Text Topic Modeling using Word Embeddings

Cited by 34 publications

References 5 publications

Machine Translation Vs. Multilingual Dictionaries Assessing Two Strategies for the Topic Modeling of Multilingual Text Collections

Machine Translation Vs. Multilingual Dictionaries Assessing Two Strategies for the Topic Modeling of Multilingual Text Collections

Enriched Latent Dirichlet Allocation for Sentiment Analysis

Leveraging Meta Information in Short Text Aggregation

Contact Info

Product

Resources

About