A survey of recent methods on deriving topics from Twitter: algorithm to evaluation

Abstract

In recent years, studies related to topic derivation in Twitter have gained a lot of interest from businesses and academics. The interconnection between users and information has made social media, especially Twitter, an ultimate platform for propagation of information about events in real time. Many applications require topic derivation from this social media platform. These include, for example, disaster management, outbreak detection, situation awareness, surveillance, and market analysis. Deriving topics from Twitter is challenging due to the short content of the individual posts. The environment itself is also highly dynamic. This paper presents a review of recent methods proposed to derive topics from social media platform from algorithms to evaluations. With regard to algorithms, we classify them based on the features they exploit, such as content, social interactions, and temporal aspects. In terms of evaluations, we discuss the datasets and metrics generally used to evaluate the methods. Finally, we highlight the gaps in the research this far and the problems that remain to be addressed.

Cross-platform comparison of framed topics in Twitter and Weibo: machine learning approaches to social media text mining

“…As social media posts are often short and may include mis-spelled words or irrelevant characters (such as emojis), social media text documents share an extremely low number of overlapping terms within a collection of posts. To address the sparsity problem, scholars have suggested alternative methods, such as LDA extension to author-topic model, and the dual LDA approach that relies on external knowledge bases like Wikipedia (Atefeh, and Khreich 2015;Nugroho et al 2020;Rafeeque and Sendhilkumar 2011).…”

“…Alternative LDA models or extensions have their own limitations when implemented in specific social media data. For example, Nugroho et al (2020) suggested that standard LDA performs, in fact, better than the author-topic model extension in the social media environment, and dual LDA only increases computational complexity. In addition, using external sources (content expansion) such as Wikipedia may present issues of reliability and uneven quality among documents when used in the dual LDA process, a problem which is amplified when comparing platforms such as Twitter and Weibo.…”

“…In addition, using external sources (content expansion) such as Wikipedia may present issues of reliability and uneven quality among documents when used in the dual LDA process, a problem which is amplified when comparing platforms such as Twitter and Weibo. In this work, we addressed the sparsity issue through extensive manual screenings on mis-spellings and emojis, and used the content merging method, by aggregating all comments generated in a certain unit of time (a week) into a single entity, as suggested by Weng et al (2010) and Nugroho et al (2020).…”

Web Platform for the Identification and Analysis of Events on Twitter

“…On the one hand, among the specialized tools it is worth mentioning that they tend to use a very technical language and, as a result, it is difficult for non-specialist users to interpret the results. As an example of this type of tools it is worth mentioning Cluto [10] and Weka [11], multi-platform tools that implement a great variety of automatic methods for data analysis. On the other hand, in the second category are tools used to analyze data that are published exclusively on social networks, and that seek to provide users, expert and inexpert, with sufficient elements to perform an easy and intuitive analysis of the results provided by their methods of identification of issues, polarity, etc.…”

Modeling Multidimensional Public Opinion Polarization Process under the Context of Derived Topics

“…There is limited study on the multi-dimensional public opinion polarization phenomenon in a topic-derived context, while most studies conduct research from a single dimension of network public opinion topics and use macro statistics [ 1 , 2 ] or a mathematical modeling method [ 3 , 4 ] to analyze the formation process. In fact, after the outbreak of an event, affected by information with multiple tuples, discussion topics with multiple dimensions tend to be derived [ 5 ]. However, netizens’ debates on topics with different dimensions will enable the polarization of online public opinions to be multi-dimensional.…”