Evaluating the Effectiveness of Deplatforming as a Moderation Strategy on Twitter

Jhaver, Shagun; Boylston, Christian; Yang, Diyi; Bruckman, Amy

doi:10.1145/3479525

Cited by 116 publications

(85 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While most researchers have tracked the activities of banned community members after [2,28,43,54,56,57,60] the ban or right before the ban [13], our analysis goes earlier in the existence of these communities to show that the majority of the Q-users likely joined Reddit to engage in and focus on QAnon-related activities. This finding extends our understanding of conspiracist behavior on Reddit, and provides a recommendation for future works to study the other aspects of the participation of active members prior to the creation of the later-banned communities, or in parallel to their activities there.…”

Section: Discussion and Limitationsmentioning

confidence: 99%

See 1 more Smart Citation

Characterizing Reddit Participation of Users Who Engage in the QAnon Conspiracy Theories

Engel,

Hua,

Zeng

et al. 2022

Preprint

View full text Add to dashboard Cite

Widespread conspiracy theories may significantly impact our society. This paper focuses on the QAnon conspiracy theory, a consequential conspiracy theory that started on and disseminated successfully through social media. Our work characterizes how Reddit users who have participated in QAnon-focused subreddits engage in activities on the platform, especially outside their own communities. Using a large-scale Reddit moderation action against QAnon-related activities in 2018 as the source, we identified 13,000 users active in the early QAnon communities. We collected the 2.1 million submissions and 10.8 million comments posted by these users across all of Reddit from October 2016 to January 2021. The majority of these users were only active after the emergence of the QAnon conspiracy theory and decreased in activity after Reddit's 2018 QAnon ban. A qualitative analysis of a sample of 915 subreddits where the "QAnon-enthusiastic" users were especially active shows that they participated in a diverse range of subreddits, often of unrelated topics to QAnon. However, most of the users' submissions were concentrated in subreddits that have sympathetic attitudes towards the conspiracy theory, characterized by discussions that were pro-Trump, or emphasized unconstricted behavior (often anti-establishment and anti-interventionist). Further study of a sample of 1,571 of these submissions indicates that most consist of links from low-quality sources, bringing potential harm to the broader Reddit community. These results point to the likelihood that the activities of early QAnon users on Reddit were dedicated and committed to the conspiracy, providing implications on both platform moderation design and future research.CCS Concepts: • Human-centered computing → Collaborative and social computing; Empirical studies in collaborative and social computing.

show abstract

Section: Discussion and Limitationsmentioning

confidence: 99%

“…The effectiveness of deplatforming. Prior work has shown that for social media websites, deplatforming is a powerful tool to reduce harmful behaviors on the platform [13,28,60]. However, users or communities that were deplatformed might move to alternative platforms [54,57] and engage in more toxic behaviors [2,55].…”

Section: Background and Related Workmentioning

confidence: 99%

Characterizing Reddit Participation of Users Who Engage in the QAnon Conspiracy Theories

Engel,

Hua,

Zeng

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…As seen in Table 3, the selected thresholds of values for identifying focus profiles lead to a small number of such profiles in all clusters, compared to the size of the 3T dataset. This can be attributed to the following reasons: a) to start with, the number of toxic tweets on Twitter is expected to be gen-erally small (∼8% was reported by Founta et al (2018)); b) then, extremely toxic profiles do not last long on Twitter: they get reported for misconduct violation and are banned fairly quickly (Jhaver et al 2021); c) our selected thresholds are conservative (min median=0.4 and max Gini=0.25) and applied in a combined fashion, as we aim to identify extreme cases of both highly toxic profiles, who are also consistent in their toxicity (not just sporadically toxic).…”

Section: Twitter Profile Clusteringmentioning

confidence: 99%

A deep dive into the consistently toxic 1% of Twitter

Qayyum¹,

Zhao²,

Wood³

et al. 2022

Preprint

View full text Add to dashboard Cite

Misbehavior in online social networks (OSN) is an evergrowing phenomenon. The research to date tends to focus on the deployment of machine learning to identify and classify types of misbehavior such as bullying, aggression, and racism to name a few. The main goal of identification is to curb natural and mechanical misconduct and make OSNs a safer place for social discourse. Going beyond past works, we perform a longitudinal study of a large selection of Twitter profiles, which enables us to characterize profiles in terms of how consistently they post highly toxic content. Our data spans 14 years of tweets from 122K Twitter profiles and more than 293M tweets. From this data, we selected the most extreme profiles in terms of consistency of toxic content and examined their tweet texts, and the domains, hashtags, and URLs they shared. We found that these selected profiles keep to a narrow theme with lower diversity in hashtags, URLs, and domains, they are thematically similar to each other (in a coordinated manner, if not through intent), and have a high likelihood of bot-like behavior (likely to have progenitors with intentions to influence). Our work contributes a substantial and longitudinal online misbehavior dataset to the research community and establishes the consistency of a profile's toxic behavior as a useful factor when exploring misbehavior as potential accessories to influence operations on OSNs.

show abstract

“…As per Grimmelmann's taxonomy of online moderation, banning is one of the four key techniques that moderators can adopt to avoid disruptions caused by malicious actors [12,19]. Several studies have focused on understanding the impact of bans and deplatforming -be it at an individual level [8,18], communitylevel (mass removal of xenophobic communities) [7,20,30,31,41], or platform level (banning of right-wing platforms like Parler from Google's app store) [2,3]. These studies find that while bans and deplatforming force a large fraction of users to abandon the platform, a few users and communities evade the ban and continue malicious activities that cause sustained issues relating to abuse, propaganda, and sometimes even leading to real-life acts of violence.…”

Section: Banning and Deplatformingmentioning

confidence: 99%

Characterizing, Detecting, and Predicting Online Ban Evasion

Niverthi,

Verma,

Kumar

2022

Preprint

View full text Add to dashboard Cite

Moderators and automated methods enforce bans on malicious users who engage in disruptive behavior. However, malicious users can easily create a new account to evade such bans. Previous research has focused on other forms of online deception, like the simultaneous operation of multiple accounts by the same entities (sockpuppetry), impersonation of other individuals, and studying the effects of de-platforming individuals and communities. Here we conduct the first data-driven study of ban evasion, i.e., the act of circumventing bans on an online platform, leading to temporally disjoint operation of accounts by the same user.We curate a novel dataset of 8, 551 ban evasion pairs (parent, child) identified on Wikipedia and contrast their behavior with benign users and non-evading malicious users. We find that evasion child accounts demonstrate similarities with respect to their banned parent accounts on several behavioral axes -from similarity in usernames and edited pages to similarity in content added to the platform and its psycholinguistic attributes. We reveal key behavioral attributes of accounts that are likely to evade bans. Based on the insights from the analyses, we train logistic regression classifiers to detect and predict ban evasion at three different points in the ban evasion lifecycle. Results demonstrate the effectiveness of our methods in predicting future evaders (𝐴𝑈𝐶 = 0.78), early detection of ban evasion (𝐴𝑈 𝐶 = 0.85), and matching child accounts with parent accounts (𝑀𝑅𝑅 = 0.97). Our work can aid moderators by reducing their workload and identifying evasion pairs faster and more efficiently than current manual and heuristic-based approaches. CCS CONCEPTS• Information systems → Web log analysis; Data mining; • Human-centered computing → Empirical studies in collaborative and social computing; • Security and privacy → Human and societal aspects of security and privacy.

show abstract

Evaluating the Effectiveness of Deplatforming as a Moderation Strategy on Twitter

Cited by 116 publications

References 48 publications

Characterizing Reddit Participation of Users Who Engage in the QAnon Conspiracy Theories

Characterizing Reddit Participation of Users Who Engage in the QAnon Conspiracy Theories

A deep dive into the consistently toxic 1% of Twitter

Characterizing, Detecting, and Predicting Online Ban Evasion

Contact Info

Product

Resources

About