An Approach to Relevancy Detection: Contributions to the Automatic Detection of Relevance in Social Networks

Abstract

In this paper we analyze the information propagated through three social networks. Previous research has shown that most of the messages posted on Twitter are truthful, but the service is also used to spread misinformation and false rumors. In this paper we focus on the search for automatic methods for assessing the relevance of a given set of posts. We first retrieved from social networks, posts related to trending topics. Then, we categorize them as being news or as being conversational messages, and assessed their credibility. From the gained insights we used features to automatically assess whether a post is news or chat, and to level its credibility. Based on these two experiments we built an automatic classifier. The results from assessing our classifier, which categorizes posts as being relevant or not, lead to a high balanced accuracy, with the potential to be further enhanced.

Predicting the Relevance of Social Media Posts Based on Linguistic Features and Journalistic Criteria

“…Considering the full feature set, MinDist achieved the best precision (0.72) while RF got the best accuracy (0.64), recall (0.78), F 1 (0.71), AUC (0.78) and AP (0.62). Although using a different dataset, these numbers already show better results than Figueira et al (2016), who achieved an accuracy of 0.59 and F 1 of 0.68.…”

“…The obtained results show a substantial improvement towards the only known work that tackled the same problem (Figueira et al, 2016), but where a much simpler feature set was exploited and a smaller dataset was used, to reach an accuracy of 0.59 and a F 1 of 0.68. Because of the different datasets used and, especially, the different classification goals and target classes, no deeper comparison can be made with related work, apart from a shallow analysis on the performance numbers reported.…”

“…To our knowledge, the closest to our work is Figueira et al (2016), who also aim to detect if social network posts are relevant (roughly, news) or irrelevant (roughly, chat). They achieve an accuracy of 0.59 and a F 1 of 0.68 in a small dataset, while exploting a small set of features, namely: (a) length of a post; (b) set of words typically used in credible posts (not reported); (c) number of occurrences of certain words; (d) excessive punctuation; (e) abundant use of smileys/emoticons.…”

“…Besides the target classes, an important difference is that we rely exclusively on linguisticallydriven features, extracted by NLP tools, which enables to predict the relevance of a post even before its publication and without any previous assumption on the author. Although with a similar goal to Figueira et al (2016), our feature set is richer, which results in a considerable improvement on the results, as it will be further reported. We should still add that, although not explicit, all their features are captured by our feature extraction approach.…”

Factual or Believable? Negotiating the Boundaries of Confirmation Bias in Online News Stories

“…Because they provided obvious context and were at the epicenter of our study, the following words were removed from the clouds shown in Figure 2: "Roy," "Moore," and "Alabama." These word clouds illustrate important context for understanding the discourse and rhetoric [36] within each time period (see Section 3.2 for descriptions of the Events and the Time Periods).…”