Do Platform Migrations Compromise Content Moderation? Evidence from r/The_Donald and r/Incels

Abstract

When toxic online communities on mainstream platforms face moderation measures, such as bans, they may migrate to other platforms with laxer policies or set up their own dedicated websites. Previous work suggests that within mainstream platforms, community-level moderation is effective in mitigating the harm caused by the moderated communities. It is, however, unclear whether these results also hold when considering the broader Web ecosystem. Do toxic communities continue to grow in terms of their user base and activity on the new platforms? Do their members become more toxic and ideologically radicalized? In this paper, we report the results of a large-scale observational study of how problematic online communities progress following community-level moderation measures. We analyze data from r/The_Donald and r/Incels, two communities that were banned from Reddit and subsequently migrated to their own standalone websites. Our results suggest that, in both cases, moderation measures significantly decreased posting activity on the new platform, reducing the number of posts, active users, and newcomers. In spite of that, users in one of the studied communities (r/The_Donald) showed increases in signals associated with toxicity and radicalization, which justifies concerns that the reduction in activity may come at the expense of a more toxic and radical community. Overall, our results paint a nuanced portrait of the consequences of community-level moderation and can inform their design and deployment.

When the Echo Chamber Shatters: Examining the Use of Community-Specific Language Post-Subreddit Ban

“…The majority of past work on bans of harmful communities on Reddit only examined one or two subreddits, often chosen due to notoriety (Hazel Kwon and Shao, 2020;Chandrasekharan et al, 2017;Saleem and Ruths, 2018;Ribeiro et al, 2020a;Habib et al, 2019;Thomas et al, 2021). Many of these studies focused on the average change in behavior across users and did not consider the factors which may drive inter-individual differences in behavior following a ban (Chandrasekharan et al, 2017;Saleem and Ruths, 2018;Habib et al, 2019).…”

“…Different users may respond differently to subreddit bans based on their level of overall activity or community engagement. For example, Ribeiro et al (2020a) found that users that were more active on Reddit prior to a subreddit ban were more likely to migrate to a different platform following a ban. A user's activity levels prior to a ban also impacted whether activity levels increased or decreased upon migrating to a different platform (Ribeiro et al, 2020a).…”

“…For example, Ribeiro et al (2020a) found that users that were more active on Reddit prior to a subreddit ban were more likely to migrate to a different platform following a ban. A user's activity levels prior to a ban also impacted whether activity levels increased or decreased upon migrating to a different platform (Ribeiro et al, 2020a). Similarly, Thomas et al (2021) demonstrated that users who were more active in a subreddit prior to a ban were more likely to change their behavior following the banning of that subreddit, but the authors did not investigate the ways in which users changed their behavior.…”

“…Online spaces often contain toxic behaviors such as abuse or harmful speech (Blackwell et al, 2017;Saleem et al, 2017;Jhaver et al, 2018;Saleem and Ruths, 2018;Habib et al, 2019;Ribeiro et al, 2020a;de Gibert et al, 2018a;Sprugnoli et al, 2018;Park and Fung, 2017;Singh et al, 2018;Lee et al, 2018). Such toxicity may result in platform-wide decreases in user participation and engagement which, combined with external pressure (e.g., bad press), may motivate platform managers to moderate harmful behavior (Saleem and Ruths, 2018;Habib et al, 2019).…”

“…Though limited, there is some evidence to suggest that subreddit bans may be effective by certain metrics. Past work has demonstrated that these bans can have both user-and community-level effects (Hazel Kwon and Shao, 2020;Chandrasekharan et al, 2017;Saleem and Ruths, 2018;Ribeiro et al, 2020a;Thomas et al, 2021;Habib et al, 2019). Several of these studies have suggested that (1) subreddit bans may lead a significant number of users to completely stop using the site, and that (2) following a ban, users that remain on the platform appear to decrease their levels of harmful speech on Reddit (Saleem and Ruths, 2018;Thomas et al, 2021;Habib et al, 2019).…”