Automatic Identification of Harmful, Aggressive, Abusive, and Offensive Language on the Web: A Survey of Technical Biases Informed by Psychology Literature

Abstract

The automatic detection of conflictual languages (harmful, aggressive, abusive, and offensive languages) is essential to provide a healthy conversation environment on the Web. To design and develop detection systems that are capable of achieving satisfactory performance, a thorough understanding of the nature and properties of the targeted type of conflictual language is of great importance. The scientific communities investigating human psychology and social behavior have studied these languages in details, but their insights have only partially reached the computer science community. In this survey, we aim both at systematically characterizing the conceptual properties of online conflictual languages, and at investigating the extent to which they are reflected in state-of-the-art automatic detection systems. Through an analysis of psychology literature, we provide a reconciled taxonomy that denotes the ensemble of conflictual languages typically studied in computer science. We then characterize the conceptual mismatches that can be observed in the main semantic and contextual properties of these languages and their treatment in computer science works; and systematically uncover resulting technical biases in the design of machine learning classification models and the dataset created for their training. Finally, we discuss diverse research opportunities for the computer science community and reflect on broader technical and structural issues.

Vicinal Risk Minimization for Few-Shot Cross-lingual Transfer in Abusive Language Detection

“…Abusive Language. Typically, abusive language refers to a wide range of concepts (Balayn et al, 2021;Poletto et al, 2021), including hate speech (Yin and Zubiaga, 2021;Alkomah and Ma, 2022;Jain and Sharma, 2022), profanity (Soykan et al, 2022), aggressive language (Muti et al, 2022;Kanclerz et al, 2021), offensive language (Pradhan et al, 2020;Kogilavani et al, 2021), cyberbullying (Rosa et al, 2019) and misogyny (Shushkevich and Cardiff, 2019). Pamungkas et al (2023) overview recent research across domains and languages.…”

Misinformation in Third-party Voice Applications

“…The other seven Skills were activated in place of the intended Skill by audio request. The Skills identified included one Skill ('Love Facts') providing misleading medical information using non-medical terminology, two Skills ('Freaky Facts' and 'Funny Facts') stating incorrectly that Chickenpox disease has been eradicated, one Skill ('My Trivia') stating subjective opinion as fact, two Skills ('Vegan Facts' and 'Teething Facts') provided health information purporting to be factual that is inconsistent with advice from official sources, 6 one Skill ('Surprising Facts') providing misleading information on suicide statistics as referring to the population as a whole rather than only one age group, and finally a Skill ('Global Warming Facts') containing unsourced and inaccurate information on climate change (the Skill stated as a definite forecast as consequence that is only stated as a possible consequence of global warming in official sources. 7 ).…”

Accounting for Offensive Speech as a Practice of Resistance

“…Tasks such as the detection of toxicity, hate speech, and online harassment have been developed to identify and intervene in situations that have the potential to cause significant social harm. These tasks for identifying and classifying offensive or undesirable language have gone by different names (see: (Waseem et al, 2017;Balayn et al, 2021)) and have employed varying task definitions, but they are united by a goal of reducing harm and breakdowns in civil discourse. Because language use varies contextually, it is difficult to model the nuanced social context that informs whether language produces harm.…”