The Datafication of Hate: Expectations and Challenges in Automated Hate Speech Monitoring

Abstract

Hate speech has been identified as a pressing problem in society and several automated approaches have been designed to detect and prevent it. This paper reports and reflects upon an action research setting consisting of multi-organizational collaboration conducted during Finnish municipal elections in 2017, wherein a technical infrastructure was designed to automatically monitor candidates' social media updates for hate speech. The setting allowed us to engage in a 2-fold investigation. First, the collaboration offered a unique view for exploring how hate speech emerges as a technical problem. The project developed an adequately well-working algorithmic solution using supervised machine learning. We tested the performance of various feature extraction and machine learning methods and ended up using a combination of Bag-of-Words feature extraction with Support-Vector Machines. However, an automated approach required heavy simplification, such as using rudimentary scales for classifying hate speech and a reliance on word-based approaches, while in reality hate speech is a linguistic and social phenomenon with various tones and forms. Second, the action-research-oriented setting allowed us to observe affective responses, such as the hopes, dreams, and fears related to machine learning technology. Based on participatory observations, project artifacts and documents, interviews with project participants, and online reactions to the detection project, we identified participants' aspirations for effective automation as well as the level of neutrality and objectivity introduced by an algorithmic system. However, the participants expressed more critical views toward the system after the monitoring process. Our findings highlight how the powerful expectations related to technology can easily end up dominating a project dealing with a contested, topical social issue. We conclude by discussing the problematic aspects of datafying hate and suggesting some practical implications for hate speech recognition.

Interdependence and Intervention: News Organizations’ Contradictory Relationship With Online Comments

“…However, some interviewees were not satisfied with the outsourced or automated moderation, mostly because of a lack of contextual knowledge. This reflects the results of previous research (e.g., Caplan 2018, Gerrard 2018, Gorwa et al 2020, Jhaver et al 2019, Laaksonen et al 2020, Tubaro et al 2020.…”

No magic bullet: estimating anti-immigrant sentiment and social desirability bias with the item-count technique

“…The increasingly ubiquitous digitization of all domains of life is opening up exciting new research options (Groves 2011;Hill et al 2020). Recent developments such as affective computing and sentiment analysis (Cambria 2016) or automated hate-speech recognition (Greevy and Smiton 2004;Laaksonen et al 2020) rely on digital traces, including social media usage, to detect sensitive attitudes such as AIS. While side-stepping traditional manifestations of response bias, such innovative data sources and research techniques are in turn vexed by various kinds of bias (Sen et al 2019), and there are no agreed procedures (yet) for deriving population estimates from such data (Japec et al 2015:872).…”

Hate speech operationalization: a preliminary examination of hate speech indicators and their structure

“…2. Online hate speech is a research field that is open to diverse knowledge-based fields, and cross-disciplinary approaches are both key and often recommended [11,[16][17][18]. While social science researchers lack knowledge on automatic hate speech detection, most data analysts have no social science background.…”