Document-based topic coherence measures for news media text

Korenčić, Damir; Ristov, Strahil; Šnajder, Jan

doi:10.1016/j.eswa.2018.07.063

Cited by 49 publications

(27 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Topic modeling was applied to tweets from both Ecig and non-Ecig groups. Topic coherence was used to determine the optimal number of topics to identify the frequently discussed topics in each group [ 17 ].…”

Section: Methodsmentioning

confidence: 99%

Electronic Cigarette Users' Perspective on the COVID-19 Pandemic: Observational Study Using Twitter Data

Gao¹,

Xie²,

Li³

2021

JMIR Public Health Surveill

View full text Add to dashboard Cite

Background Previous studies have shown that electronic cigarette (e-cigarette) users might be more vulnerable to COVID-19 infection and could develop more severe symptoms if they contract the disease owing to their impaired immune responses to viral infections. Social media platforms such as Twitter have been widely used by individuals worldwide to express their responses to the current COVID-19 pandemic. Objective In this study, we aimed to examine the longitudinal changes in the attitudes of Twitter users who used e-cigarettes toward the COVID-19 pandemic, as well as compare differences in attitudes between e-cigarette users and nonusers based on Twitter data. Methods The study dataset containing COVID-19–related Twitter posts (tweets) posted between March 5 and April 3, 2020, was collected using a Twitter streaming application programming interface with COVID-19–related keywords. Twitter users were classified into two groups: Ecig group, including users who did not have commercial accounts but posted e-cigarette–related tweets between May 2019 and August 2019, and non-Ecig group, including users who did not post any e-cigarette–related tweets. Sentiment analysis was performed to compare sentiment scores towards the COVID-19 pandemic between both groups and determine whether the sentiment expressed was positive, negative, or neutral. Topic modeling was performed to compare the main topics discussed between the groups. Results The US COVID-19 dataset consisted of 4,500,248 COVID-19–related tweets collected from 187,399 unique Twitter users in the Ecig group and 11,479,773 COVID-19–related tweets collected from 2,511,659 unique Twitter users in the non-Ecig group. Sentiment analysis showed that Ecig group users had more negative sentiment scores than non-Ecig group users. Results from topic modeling indicated that Ecig group users had more concerns about deaths due to COVID-19, whereas non-Ecig group users cared more about the government’s responses to the COVID-19 pandemic. Conclusions Our findings show that Twitter users who tweeted about e-cigarettes had more concerns about the COVID-19 pandemic. These findings can inform public health practitioners to use social media platforms such as Twitter for timely monitoring of public responses to the COVID-19 pandemic and educating and encouraging current e-cigarette users to quit vaping to minimize the risks associated with COVID-19.

show abstract

Section: Methodsmentioning

confidence: 99%

Electronic Cigarette Users' Perspective on the COVID-19 Pandemic: Observational Study Using Twitter Data

Gao¹,

Xie²,

Li³

2021

JMIR Public Health Surveill

View full text Add to dashboard Cite

show abstract

“…Here topics are generated before some form of qualitative method is used to gain insights into the data. These methods include exploratory content analysis (Korenčić et al, 2018), critical discourse analysis (Törnberg and Törnberg, 2016), digital autoethnography (Brown, 2019), grounded theory (Baumer et al, 2017), and thematic analysis (Doogan et al, 2020;Andreotta et al, 2019).…”

Section: Practical Applicationsmentioning

confidence: 99%

“…Alternatively, the T dc alone cam be used for in-depth analysis (Törnberg and Törnberg, 2016). However, human evaluation tasks that require open labeling are not generally used to validate new coherence measures (O'Callaghan et al, 2015;Korenčić et al, 2018).…”

Section: Practical Applicationsmentioning

confidence: 99%

Topic Model or Topic Twaddle? Re-evaluating Semantic Interpretability Measures

Doogan¹,

Buntine²

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

When developing topic models, a critical question that should be asked is: How well will this model work in an applied setting? Because standard performance evaluation of topic interpretability uses automated measures modeled on human evaluation tests that are dissimilar to applied usage, these models' generalizability remains in question. In this paper, we probe the issue of validity in topic model evaluation and assess how informative coherence measures are for specialized collections used in an applied setting. Informed by the literature, we propose four understandings of interpretability. We evaluate these using a novel experimental framework reflective of varied applied settings, including human evaluations using open labeling, typical of applied research. These evaluations show that for some specialized collections, standard coherence measures may not inform the most appropriate topic model or the optimal number of topics, and current interpretability performance validation methods are challenged as a means to confirm model quality in the absence of ground truth data.

show abstract

“…Evaluating the impact of mass media requires rapid processing of large amounts of textual information, which can be achieved using natural language processing (NLP) and machine learning (machine learning -ML) techniques. These technologies allow users to extract information from large amounts of textual data [1,2], provide content analysis [3,4], personalized access to news [5][6][7], and even support its production and distribution [8,9].…”

Section: Summary (Required)mentioning

confidence: 99%

KazNewsDataset: Single Country Overall Digital Mass Media Publication Corpus

Yakunin¹,

Kalimoldayev²,

Mukhamediev³

et al. 2021

Data

View full text Add to dashboard Cite

Mass media is one of the most important elements influencing the information environment of society. The mass media is not only a source of information about what is happening but is often the authority that shapes the information agenda, the boundaries, and forms of discussion on socially relevant topics. A multifaceted and, where possible, quantitative assessment of mass media performance is crucial for understanding their objectivity, tone, thematic focus and, quality. The paper presents a corpus of Kazakhstan media, which contains over 4 million publications from 36 primary sources (which has at least 500 publications). The corpus also includes more than 2 million texts of Russian media for comparative analysis of publication activity of the countries, also about 4000 sections of state policy documents. The paper briefly describes the natural language processing and multiple-criteria decision-making methods, which are the algorithmic basis of the text and mass media evaluation method, and describes the results of several research cases, such as identification of propaganda, assessment of the tone of publications, calculation of the level of socially relevant negativity, comparative analysis of publication activity in the field of renewable energy. Experiments confirm the general possibility of evaluating the socially significant news, identifying texts with propagandistic content, evaluating the sentiment of publications using the topic model of the text corpus since the area under receiver operating characteristics curve (ROC AUC) values of 0.81, 0.73 and 0.93 were achieved on abovementioned tasks. The described cases do not exhaust the possibilities of thematic, tonal, dynamic, etc., analysis of the considered corpus of texts. The corpus will be interesting to researchers considering both multiple publications and mass media analysis, including comparative analysis and identification of common patterns inherent in the media of different countries.

show abstract

Document-based topic coherence measures for news media text

Cited by 49 publications

References 55 publications

Electronic Cigarette Users' Perspective on the COVID-19 Pandemic: Observational Study Using Twitter Data

Electronic Cigarette Users' Perspective on the COVID-19 Pandemic: Observational Study Using Twitter Data

Topic Model or Topic Twaddle? Re-evaluating Semantic Interpretability Measures

KazNewsDataset: Single Country Overall Digital Mass Media Publication Corpus

Contact Info

Product

Resources

About