The fabrics of machine moderation: Studying the technical, normative, and organizational structure of Perspective API

Abstract

Over recent years, the stakes and complexity of online content moderation have been steadily raised, swelling from concerns about personal conflict in smaller communities to worries about effects on public life and democracy. Because of the massive growth in online expressions, automated tools based on machine learning are increasingly used to moderate speech. While ‘design-based governance’ through complex algorithmic techniques has come under intense scrutiny, critical research covering algorithmic content moderation is still rare. To add to our understanding of concrete instances of machine moderation, this article examines Perspective API, a system for the automated detection of ‘toxicity’ developed and run by the Google unit Jigsaw that can be used by websites to help moderate their forums and comment sections. The article proceeds in four steps. First, we present our methodological strategy and the empirical materials we were able to draw on, including interviews, documentation, and GitHub repositories. We then summarize our findings along five axes to identify the various threads Perspective API brings together to deliver a working product. The third section discusses two conflicting organizational logics within the project, paying attention to both critique and what can be learned from the specific case at hand. We conclude by arguing that the opposition between ‘human’ and ‘machine’ in speech moderation obscures the many ways these two come together in concrete systems, and suggest that the way forward requires proactive engagement with the design of technologies as well as the institutions they are embedded in.

Making the Car “Platform Ready”: How Big Tech Is Driving the Platformization of Automobility

“…To derive these levels, we draw upon a variety of empirical materials, including scientific and technical literature from across fields and disciplines, industry reports and white papers, product and development reference documentation, promotional materials, and online videos. In this, we employ a “technographic” approach (Bucher, 2018; Rieder & Skop, 2021) to draw out the critical techniques and devices through which platformization is occurring. In some cases, platform discourse is used by some of the actors themselves (e.g., scientists, engineers, and companies) while in other cases, we encounter more subtle technological or material characteristics that can be identified as part of platformization.…”

Dataset — Make Reddit Great Again: Assessing Community Effects of Moderation Interventions on r/The_Donald

“…Comment toxicity. Our indicator of toxicity is based on the severe toxicity score provided by the well-known Google Perspective API (Rieder and Skop 2021), which was in part trained based on Reddit comments. Trustworthiness of shared news.…”

Building socially responsible conversational agents using big data to support online learning: A case with Algebra Nation

“…Perspective API has been trusted by commercial products such as the New York Times and Disqus (Perspective API, 2021b). Numerous researchers have also examined Perspective API to perform automatic text safety detection and found the product robust and accurate (eg, Obadimu et al, 2019; Rieder & Skop, 2021). In this study, we will utilize Perspective API to evaluate CA content offensiveness to assist in the examination of the three safety dimensions.…”