COLD: Annotation scheme and evaluation data set for complex offensive language in English

Abstract

This paper presents a new, extensible annotation scheme for offensive language data sets. The annotation scheme expands coverage beyond fairly straightforward cases of offensive language to address several cases of complex, implicit, and/or pragmaticallytriggered offensive language. We apply the annotation scheme to create a new Complex Offensive Language Data Set for English (COLD-EN). The primary purpose of this data set is to diagnose how well systems for automatic detection of abusive language are able to classify three types of complex offensive language: reclaimed slurs, offensive utterances containing pejorative adjectival nominalizations (and no slur terms), and utterances conveying offense through linguistic distancing.COLD offers a straightforward framework for error analysis. Our vision is that researchers will use this data set to diagnose the strengths and weaknesses of their offensive language detection systems. In this paper, we diagnose some strengths and weaknesses of a top-performing offensive language detection system by: a) using it to classify COLD, and b) investigating its performance on the 10 fine-grained categories supported by our annotation scheme. We evaluate the system's performance when trained on five different standard data sets for offensive language detection. Systems trained on different data sets have different strengths and weaknesses, with most performing poorly on the phenomena of reclaimed slurs and pejorative nominalizations. NOTE: This paper contains sensitive and offensive material. The offensive materials are part of a complex puzzle we wish to better understand; they appear in the form of lightly-censored slurs and degrading insults. We do not condone this type of language, nor does it reflect the attitudes or beliefs of the authors.

Proceedings of the 1st GenBench Workshop on (Benchmarking) Generalisation in NLP

“…Nonetheless, non-i.i.d. generalisation is a persisting challenge (Yin and Zubiaga, 2021b), because models tend to overfit to specific topics (Nejadgholi and Kiritchenko, 2020;Bourgeade et al, 2023), social media users (Arango et al, 2019), or keywords, such as slurs or pejorative terms (Dixon et al, 2018;Kennedy et al, 2020;Talat et al, 2018;Palmer et al, 2020;Kurrek et al, 2020). When such overt terms are missing, models often fail to detect hate speech (ElSherief et al, 2021).…”

Latent Feature-based Data Splits to Improve Generalisation Evaluation: A Hate Speech Detection Case Study

“…Nonetheless, non-i.i.d. generalisation is a persisting challenge (Yin and Zubiaga, 2021b), because models tend to overfit to specific topics (Nejadgholi and Kiritchenko, 2020;Bourgeade et al, 2023), social media users (Arango et al, 2019), or keywords, such as slurs or pejorative terms (Dixon et al, 2018;Kennedy et al, 2020;Talat et al, 2018;Palmer et al, 2020;Kurrek et al, 2020). When such overt terms are missing, models often fail to detect hate speech (ElSherief et al, 2021).…”

Addressing religious hate online: from taxonomy creation to automated detection

“…Other taxonomies include more fine-grained categories of abuse. For example, the annotation scheme proposed by Palmer et al (2020) revolves around the offensiveness of a message, the presence of slurs, adjectival nominalization, and distancing. Sanguinetti et al (2018) , instead, focus on hate messages against immigrants and annotate hate intensity, aggressiveness, offensiveness, irony and stereotypes.…”

“…Since abusive language online is a multi-faceted phenomenon and different categorizations of hate speech have been proposed over the years to account for different targets ( Kumar et al, 2018 ; Vidgen & Yasseri, 2020 ; Palmer et al, 2020 ; Zampieri et al, 2020 ), we aim on the one hand to take the complexity of online abuse into account, and on the other hand to be compatible with existing annotation schemes as much as possible. We therefore build upon the hierarchical taxonomy proposed in Zeinert, Inie & Derczynski (2021) , which has been designed to annotate misogynist messages online but nevertheless provides a backbone for the fine-grained annotation of other target types.…”