Safe from “harm”: The governance of violence by platforms

Abstract

A number of issues have emerged related to how platforms moderate and mitigate “harm.” Although platforms have recently developed more explicit policies in regard to what constitutes “hate speech” and “harmful content,” it appears that platforms often use subjective judgments of harm that specifically pertains to spectacular, physical violence—but harm takes on many shapes and complex forms. The politics of defining “harm” and “violence” within these platforms are complex and dynamic, and represent entrenched histories of how control over these definitions extends to people's perceptions of them. Via a critical discourse analysis of policy documents from three major platforms (Facebook, Twitter, and YouTube), we argue that platforms' narrow definitions of harm and violence are not just insufficient but result in these platforms engaging in a form of symbolic violence. Moreover, the platforms position harm as a floating signifier, imposing conceptions of not just what violence is and how it manifests, but who it impacts. Rather than changing the mechanisms of their design that enable harm, the platforms reconfigure intentionality and causality to try to stop users from being “harmful,” which, ironically, perpetuates harm. We provide a number of suggestions, namely a restorative justice‐focused approach, in addressing platform harm.

Humour as an online safety issue: Exploring solutions to help platforms better address this form of expression

“…The EU and UK are just two of a growing number of jurisdictions (Linklaters, 2021) pioneering regulation with the stated aims of pushing platforms to protect free expression whilst also transparently and consistently addressing the spread of harmful content and behaviour. The challenge is that the notion of 'harm' , much less ' online harm' , is not self-evident, and important critiques have focused on how '[online] harm' is being conceptualised by platforms (DeCook et al, 2022) and by those pushing for measures to address it (Nash, 2019a;Turillazzi et al, 2022, p. 10). The question of definition and scope has proven to be particularly contentious 8 within jurisdictions like the UK, which wanted to regulate not only illegal content and conduct that causes harm (e.g., speech that harasses or incites violence), but also 'legal but harmful' content and activity.…”

The Internet regulation turn? Policy, Internet and technology

“…DeCook, Cotter, Kanthawala and Foyle, in Safe From ‘Harm’: The Governance of Violence by Platforms (2021), explores how platforms, in a bid to mitigate ‘harm’ are ‘powerfully shape[ing] normative notions of harm and violence, effectively managing perceptions of their actions and directing users’ understanding of what is “harmful” and what is not’. This article contributes to the emerging work about the significance of user agency within these conversations, by highlighting the difficulties of specific communication nuances across social media platforms especially.…”

Modulating moderation: A history of objectionability in Twitter moderation practices

“…The latter, for example, has been subject to a recent study by Rogers (Rogers, 2020), who traces the migration of "extreme influencers" from Twitter to Telegram using user status metrics and content engagement available through both platforms' APIs. Studies on content moderation rely heavily on close readings of contemporary versions of content moderation policies, with few exceptions relying on versions archived by the Wayback Machine for longitudinal studies (see DeCook et al, 2022). These approaches have yet to be systematically combined, and are still prey to changing APIs and reprisals from platforms cramping down on researchers who must scrape information not accessible otherwise (Bond, 2021).…”

“…Hateful Conduct Galtung (1990) argues that cultural violence-which can be based on religion, ideology, and science, to name only a few categories he covers-is intimately linked with structural and direct violence, referring to it as "a substratum from which the other two can derive their nutrients" (see also DeCook et al, 2022). However, hateful conduct is not usually legally circumscribed, except in specific cases (eg.…”

“…This article contributes to this scholarship by answering the following question: how have platforms' ideas and moderation of objectionable information, particularly violent speech, changed over time? We use web history (Brügger, 2010) and other digital methods (Rogers, 2013) to propose a "digital forensics" that retrieves the traces of updated, overwritten, and removed content to develop a genealogy of violent speech, which we refer to as an ensemble of abusive, hateful, graphic, incitement and other related content (DeCook et al, 2022). This category of objectionable information is central to any practical operationalization of content moderation, particularly Twitter's policies and enforcement techniques against a variety of social and institutional threats.…”