Comparison between LDA &amp; NMF for event-detection from large text stream data

Suri, Pranav; Roy, Nihar Ranjan

doi:10.1109/ciact.2017.7977281

Cited by 36 publications

(15 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They categorized detection algorithms into five types, including clustering, frequent pattern mining, exemplar-based technique, matrix factorization, and probabilistic models. Among these different kinds of algorithms, both probabilistic models and matrix factorization attract the most attention [7,12,14,16,24,25,29].…”

Section: Algorithms For Topic Detection In Social Mediamentioning

confidence: 99%

See 1 more Smart Citation

What Are People Concerned About During the Pandemic? Detecting Evolving Topics about COVID-19 from Twitter

Chang

Monselise

Yang

2021

J Healthc Inform Res

View full text Add to dashboard Cite

With the novel coronavirus (COVID-19) pandemic affecting the lives of the citizens of over 200 countries, there is a need for policy makers and clinicians to understand public sentiment and track the spread of the disease. One of the sources for gaining valuable insight into public sentiment is through social media. This study aims to extract this insight by producing a list of the most discussed topics regarding COVID-19 on Twitter every week and monitoring the evolution of topics from week to week. This research will propose two topic mining that can handle a large-scale datasetrolling online non-negative matrix factorization (Rolling-ONMF) and sliding online non-negative matrix factorization (Sliding-ONMF)-and compare the insights produced by both techniques. Each algorithm produces 425 topics over the course of 17 weeks. However, topics that have not evolved from one week to the next beyond a certain evolution threshold are consolidated into a single topic. Since the topics produced by the Rolling-ONMF algorithm each week depend on the topics from the previous week, we find that the Sliding-ONMF algorithm produces more varied topics each week; however, the topics produced by the Rolling-ONMF algorithm contain keywords that appear more consistent with each other when reviewing the terms manually. We also observe that the Sliding-ONMF algorithm is able to capture events that have shorter time frames rather than ones that last throughout many months while the Rolling-ONMF algorithm detects more general themes due to a higher average evolution score which leads to more topic consolidation. We have also conducted a qualitative analysis and grouped the detected topics into themes. A number of important themes such as government policy, economic crisis, COVID-19-related updates,

show abstract

Section: Algorithms For Topic Detection In Social Mediamentioning

confidence: 99%

“…Likewise, Suri and Roy compared the use of LDA and NMF in topic detection in tweet collections and found that LDA takes longer to run. Therefore, in real-time scenarios, NMF is preferred for Twitter topic detection [25].…”

Section: Algorithms For Topic Detection In Social Mediamentioning

confidence: 99%

What Are People Concerned About During the Pandemic? Detecting Evolving Topics about COVID-19 from Twitter

Chang

Monselise

Yang

2021

J Healthc Inform Res

View full text Add to dashboard Cite

show abstract

“…Non-negative matrix factorization (NMF) is a linear-algebraic optimization algorithm [9] used for dimensionality reduction and data analysis [10] that solves the following problem (illustrated in Fig 2, which is taken from [11]):…”

Section: Nmfmentioning

confidence: 99%

Topic Modeling Coherence: A Comparative Study between LDA and NMF Models using COVID’19 Corpus

Mifrah¹,

Benlahmar²

2020

IJATCSE

View full text Add to dashboard Cite

Topic modeling is a method for finding abstract topics in a large collection of documents. With it, it is possible to discover the mixture of hidden or "latent" topics that varies from document to document in a given corpus. As an unsupervised machine learning approach, topic models are not easy to evaluate since there is no labelled "ground truth" data to compare with. However, since topic modeling typically requires defining some parameters beforehand (first and foremost the number of topics k to be discovered), model evaluation is crucial in order to find an "optimal" set of parameters for the given data. Latent Dirichlet allocation (LDA) and Non-Negative Matrix Factorization (NMF) are the two most popular topic modeling techniques. LDA uses a probabilistic approach where as NMF uses matrix factorization approach. In this paper we want to assess which most relevant technique for topic coherence using c_v measure, we have chosen citations's Covid'19 Corpus for experimentations.

show abstract

“…This approach is basically very useful for search engines, to automate the customer service and other areas where knowing the topics from texts is crucial. There are several algorithms available that be trained for topic modelling such as LDA, LSA, NMF and Clustering [34][35][36][37][38][39][40][41][42][43][44]. These algorithms are unsupervised methods, which means, the relationship among document is not revealed prior to the model being executed.…”

Section: Topic Modellingmentioning

confidence: 99%

Exploring the Non-Medical impacts of Covid-19 using Natural Language Processing

Agade¹,

Balpande²

2020

Preprint

View full text Add to dashboard Cite

Ongoing COVID-19 Pandemic has resulted into massive damage to various platforms of global economy which has caused disruption to human livelihood. Natural Language Processing has been extensively used in different organizations to categorize sentiments, perform recommendation, summarizing information and topic modelling. This research aims to understand the non-medical impact of COVID-19 on global economy by leveraging the natural language processing methodology. This methodology comprises of text classification which includes topic modelling on unstructured COVID-19 media articles dataset provided by Anacode. Like other Natural Language Processing algorithms, Latent Dirichlet allocation (LDA) and Non-negative matrix factorization (NMF) has been proposed to classify the media articles dataset in order to analyze COVID-19 pandemic impacts in the different sectors of global economy. Model Accuracy was examined based on the coherence and perplexity score which came out to be 0.51 and -10.90 using LDA algorithm. Both the LDA and NMF algorithm identified similar prevalent topics that was impacted by COVID-19 pandemic in multiple sectors of economy. Through intertopic distance map visualization produced by LDA algorithm, it can be reciprocated that general industries which includes children schooling, parental care, and family gatherings had the major impact followed by business sector and the financial industry.

show abstract

Comparison between LDA & NMF for event-detection from large text stream data

Cited by 36 publications

References 2 publications

What Are People Concerned About During the Pandemic? Detecting Evolving Topics about COVID-19 from Twitter

What Are People Concerned About During the Pandemic? Detecting Evolving Topics about COVID-19 from Twitter

Topic Modeling Coherence: A Comparative Study between LDA and NMF Models using COVID’19 Corpus

Exploring the Non-Medical impacts of Covid-19 using Natural Language Processing

Contact Info

Product

Resources

About