eventAI at SemEval-2019 Task 7: Rumor Detection on Social Media by Exploiting Content, User Credibility and Propagation Information

Abstract

This paper describes our system for SemEval 2019 RumorEval: Determining rumor veracity and support for rumors (SemEval 2019 Task 7). This track has two tasks: Task A is to determine a user's stance towards the source rumor, and Task B is to detect the veracity of the rumor: true, false or unverified. For stance classification, a neural network model with language features is utilized. For rumor verification, our approach exploits information from different dimensions: rumor content, source credibility, user credibility, user stance, event propagation path, etc. We use an ensemble approach in both tasks, which includes neural network models as well as the traditional classification algorithms. Our system is ranked 1 st place in the rumor verification task by both the macro F1 measure and the RMSE measure.

Rumor Detection on Social Media: Datasets, Methods and Opportunities

“…Other studies utilizing propagation path are (Kwon et al, 2017;Wu et al, 2015;Chen et al, 2016;Yang et al, 2018;Li et al, 2019a;Li et al, 2019b). Experiments from these studies show that models employing propagation path perform better than the feature-based algorithms.…”

“…Most studies use four stance categories: supporting, denying, querying and commenting. Some studies have explicitly used stance information in their rumor detection model, and have shown big performance improvement (Liu et al, 2015;Enayet and El-Beltagy, 2017;Ma et al, 2018a;Kochkina et al, 2018), including the two systems, (Enayet and El-Beltagy, 2017) and (Li et al, 2019a), that were ranked No. 1 in SemEval 2017 and SemEval 2019 rumor detection tasks, respectively.…”

“…An early study from (Castillo et al, 2011) uses many text features in their model, such as the fraction of tweets with hashtags. These features and other additional text features are also used in other studies (Liu et al, 2015;Enayet and El-Beltagy, 2017;Li et al, 2019a;Ma et al, 2017;Li et al, 2019b). Kwon et al (2013) (Gupta et al, 2013;Jin et al, 2017a;Jin et al, 2017b;Shu et al, 2017a;Liu et al, 2015;Li et al, 2019a;Li et al, 2019b;Shu et al, 2017).…”

“…Users engage in rumor dissemination in multiple ways, such as sharing, liking, forwarding and reviewing. Many previous studies have shown that user credibility information is very important in rumor verification (Castillo et al, 2011;Yang et al, 2012;Gupta et al, 2012;Liu et al, 2015;Vosoughi, 2015;Shu et al, 2017b;Zhang et al, 2018;Liu and Wu, 2018;Li et al, 2019a;Li et al, 2019b). Based on 421 false rumors and 1.47 million related tweets, Li et al (2016) study various semantic aspects of false rumors, and analyze their spread and user characteristics.…”

“…Hand-crafted user features like registration age of users, number of followers, the number of posts the user had authored, and the like, are leveraged along with other textual and propagation features in Castillo et al (2011) and other studies (Liu et al, 2015;Enayet and El-Beltagy, 2017;Li et al, 2019a;Li et al, 2019b). Liu and Wu (2018) construct user representations using network embedding approaches on the social network graph.…”