The Ideal Topic: Interdependence of Topic Interpretability and Other Quality Features in Topic Modelling for Short Texts

Blekanov, Ivan S.; Bodrunova, Svetlana S.; Zhuravleva, Nina N.; Smoliarova, Anna S.; Tarasov, Nikita

doi:10.1007/978-3-030-49570-1_2

Cited by 5 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, we need to note that our prior experiments with topic models and human coding have shown that manual coding should not be taken as baseline. In particular, for pairs of human coders, topic interpretability and proximity varied for over 10 times, depending on prior familiarity with the dataset and coding experience [33]. Thus, the quality assessment of clustering remains an issue.…”

Section: Discussionmentioning

confidence: 99%

Topic Detection Based on Sentence Embeddings and Agglomerative Clustering with Markov Moment

et al. 2020

Self Cite

View full text Add to dashboard Cite

The paper is dedicated to solving the problem of optimal text classification in the areaof automated detection of typology of texts. In conventional approaches to topicality-based textclassification (including topic modeling), the number of clusters is to be set up by the scholar, andthe optimal number of clusters, as well as the quality of the model that designates proximity oftexts to each other, remain unresolved questions. We propose a novel approach to the automateddefinition of the optimal number of clusters that also incorporates an assessment of word proximityof texts, combined with text encoding model that is based on the system of sentence embeddings.Our approach combines Universal Sentence Encoder (USE) data pre-processing, agglomerativehierarchical clustering by Ward’s method, and the Markov stopping moment for optimal clustering.The preferred number of clusters is determined based on the “e-2” hypothesis. We set up anexperiment on two datasets of real-world labeled data: News20 and BBC. The proposed model istested against more traditional text representation methods, like bag-of-words and word2vec, to showthat it provides a much better-resulting quality than the baseline DBSCAN and OPTICS models withdifferent encoding methods. We use three quality metrics to demonstrate that clustering quality doesnot drop when the number of clusters grows. Thus, we get close to the convergence of text clusteringand text classification.

show abstract

Section: Discussionmentioning

confidence: 99%

Topic Detection Based on Sentence Embeddings and Agglomerative Clustering with Markov Moment

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, the categorizations produced by these models can be difficult to interpret by humans, limiting their usefulness (Blekanov et al, 2020).…”

Section: Limitations Of Current Methodsmentioning

confidence: 99%

Bridging the gap between supervised classification and unsupervised topic modelling for social-media assisted crisis management

Brunila,

Zhao,

Mircea

et al. 2021

Preprint

View full text Add to dashboard Cite

Social media such as Twitter provide valuable information to crisis managers and affected people during natural disasters. Machine learning can help structure and extract information from the large volume of messages shared during a crisis; however, the constantly evolving nature of crises makes effective domain adaptation essential. Supervised classification is limited by unchangeable class labels that may not be relevant to new events, and unsupervised topic modelling by insufficient prior knowledge. In this paper, we bridge the gap between the two and show that BERT embeddings finetuned on crisis-related tweet classification can effectively be used to adapt to a new crisis, discovering novel topics while preserving relevant classes from supervised training, and leveraging bidirectional self-attention to extract topic keywords. We create a dataset of tweets from a snowstorm to evaluate our method's transferability to new crises, and find that it outperforms traditional topic models in both automatic, and human evaluations grounded in the needs of crisis managers. More broadly, our method can be used for textual domain adaptation where the latent classes are unknown but overlap with known classes from other domains.

show abstract

“…Previous studies that used Twitter data for topic modeling to solve communication science tasks, e.g., for agenda detection purposes or finding pivotal moments in networked discussions [11], have had only moderate success. Moreover, social media data, and especially the noisy, unstructured, and extremely short texts from microblogs such as Twitter, have created multiple complications for topicality, sentiment, and opinion detection [57]. This implies that other methods, such as text summarization, the success of which does not depend that much on text length, may be applied to discussion evolution studies with better results.…”

Section: Post Title Post Summary Comment Summarymentioning

confidence: 99%

Transformer-Based Abstractive Summarization for Reddit and Twitter: Single Posts vs. Comment Pools in Three Languages

2022

Self Cite

View full text Add to dashboard Cite

Abstractive summarization is a technique that allows for extracting condensed meanings from long texts, with a variety of potential practical applications. Nonetheless, today’s abstractive summarization research is limited to testing the models on various types of data, which brings only marginal improvements and does not lead to massive practical employment of the method. In particular, abstractive summarization is not used for social media research, where it would be very useful for opinion and topic mining due to the complications that social media data create for other methods of textual analysis. Of all social media, Reddit is most frequently used for testing new neural models of text summarization on large-scale datasets in English, without further testing on real-world smaller-size data in various languages or from various other platforms. Moreover, for social media, summarizing pools of texts (one-author posts, comment threads, discussion cascades, etc.) may bring crucial results relevant for social studies, which have not yet been tested. However, the existing methods of abstractive summarization are not fine-tuned for social media data and have next-to-never been applied to data from platforms beyond Reddit, nor for comments or non-English user texts. We address these research gaps by fine-tuning the newest Transformer-based neural network models LongFormer and T5 and testing them against BART, and on real-world data from Reddit, with improvements of up to 2%. Then, we apply the best model (fine-tuned T5) to pools of comments from Reddit and assess the similarity of post and comment summarizations. Further, to overcome the 500-token limitation of T5 for analyzing social media pools that are usually bigger, we apply LongFormer Large and T5 Large to pools of tweets from a large-scale discussion on the Charlie Hebdo massacre in three languages and prove that pool summarizations may be used for detecting micro-shifts in agendas of networked discussions. Our results show, however, that additional learning is definitely needed for German and French, as the results for these languages are non-satisfactory, and more fine-tuning is needed even in English for Twitter data. Thus, we show that a ‘one-for-all’ neural-network summarization model is still impossible to reach, while fine-tuning for platform affordances works well. We also show that fine-tuned T5 works best for small-scale social media data, but LongFormer is helpful for larger-scale pool summarizations.

show abstract

The Ideal Topic: Interdependence of Topic Interpretability and Other Quality Features in Topic Modelling for Short Texts

Cited by 5 publications

References 6 publications

Topic Detection Based on Sentence Embeddings and Agglomerative Clustering with Markov Moment

Topic Detection Based on Sentence Embeddings and Agglomerative Clustering with Markov Moment

Bridging the gap between supervised classification and unsupervised topic modelling for social-media assisted crisis management

Transformer-Based Abstractive Summarization for Reddit and Twitter: Single Posts vs. Comment Pools in Three Languages

Contact Info

Product

Resources

About