Handling Disagreement in Hate Speech Modelling

Abstract

Hate speech annotation for training machine learning models is an inherently ambiguous and subjective task. In this paper, we adopt a perspectivist approach to data annotation, model training and evaluation for hate speech classification. We first focus on the annotation process and argue that it drastically influences the final data quality. We then present three large hate speech datasets that incorporate annotator disagreement and use them to train and evaluate machine learning models. As the main point, we propose to evaluate machine learning models through the lens of disagreement by applying proper performance measures to evaluate both annotators’ agreement and models’ quality. We further argue that annotator agreement poses intrinsic limits to the performance achievable by models. When comparing models and annotators, we observed that they achieve consistent levels of agreement across datasets. We reflect upon our results and propose some methodological and ethical considerations that can stimulate the ongoing discussion on hate speech modelling and classification with disagreement.

Enhancing Hate Speech Detection with Fine-Tuned Large Language Models Requires High-Quality Data

“…The by far lowest level of inter-coder agreement is found for crowd workers from Appen (α = 0.080). The overall relatively low inter-coder agreement within all groups is in line with existing literature, emphasizing the inherent challenges in achieving consensus among annotators for subjective tasks like hate speech detection (Akhtar, Basile, & Patti, 2020;Kralj Novak et al, 2022).…”

Online conspiracy communities are more resilient to deplatforming

“…Quantifying toxic messages is relevant to assess whether a less regulated environment induces an increase in the use of toxic speech. To this aim, we classify each message with the IMSyPP classifier ( 39 ) (see Methods). Figure 3 (bottom row) shows a QQ comparison of the fraction of toxic messages between platforms.…”

“…In order to detect toxic comments, we used the IMSyPP classifier ( 39 ) (publicly available d ) to label each comment on Reddit and Voat as toxic or not. We define a message as toxic if it is classified into one of the following categories: (i) inappropriate , the message contains terms that are obscene or vulgar, but the text is not directed to any person or group specifically; (ii) offensive , the comment includes offensive generalization, contempt, dehumanization, or indirect offensive remarks; or (iii) violent , the comment’s author threatens, indulges, desires or calls for physical violence against a target; it also includes calling for, denying or glorifying war crimes and crimes against humanity.…”

“…If the message is classified in the remaining category, namely appropriate , it is considered nontoxic. The accuracy for toxicity classification of the IMSyPP classifier is 0.84 ( 39 ). As a test of robustness, we contrast the results of this classifier with those of the Google Perspective API, whose toxicity score is in [0,1].…”

Toward a Perspectivist Turn in Ground Truthing for Predictive Computing

“…Akhtar, Basile, and Patti (2020) showed that a strong perspectivist approach to model training may also lead to performance improvements. Similarly, Kocoń et al (2021) proposed to leverage non-aggregated data to train models adapted to different users, in what they call "humancentered approach"; Sudre et al ( 2019), Gordon et al (2022) and Guan et al (2018) proposed multi-task approaches to deal with observer variability and dissenting voices, showing how jointly learning the consensus process and the individual raters' labels improves classification accuracy and representation; Sachdeva et al (2022a) and Kralj Novak et al (2022) showed how accounting for disagreements among raters may more accurately represent performance of ML models in hate speech detection and also improve the identification of target groups; similarly, Rodrigues and Pereira (2018) proposed a novel deep learning model that by internally capturing the reliability and biases of different annotators achieves state-of-the-art results for various crowdsourced datasets; Peterson et al (2019) showed that accounting for raters' disagreement and uncertainty may lead to generalizability and performance improvements in CV tasks; Uma et al (2020) proposed the use of soft losses as a perspectivist approach for the training of ML models in NLP tasks, while Campagner et al (2021) proposed a soft loss ensemble learning method, inspired by possibility theory and three-way decisions, for the training of ML models in perspectivist settings; similarly, Washington et al (2021) showed how the use of soft-labels, that is distributions over labels obtained by means of crowdsourcing, could be useful to better account for the subjectivity of human interpretation in emotion recognition tasks.…”