Analyzing Hate Speech Data along Racial, Gender and Intersectional Axes

Maronikolakis, Antonis; Philip, Baader,; Schütze, Hinrich

doi:10.18653/v1/2022.gebnlp-1.1

Cited by 6 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A significant body of work that has examined intersectional hate detection has, for the most part, concentrated on addressing intersectional bias in hate speech datasets Maronikolakis et al (2022); Rankin and Thomas (2020); Kim (2020). (Chandra et al 2021) (Fitzsimons 2022) examined the quantification of intersectional injustice across several demographic groups on Twitter and discovered that the collection of intersectional data is grossly inadequate, and NLP is merely a piece in inherent biases in intersectional hate detections.…”

Section: Intersectional Hate Detectionmentioning

confidence: 99%

Misogynoir: challenges in detecting intersectional hate

Kwarteng

Perfumi

Farrell

et al. 2022

Soc. Netw. Anal. Min.

View full text Add to dashboard Cite

Abstract“Misogynoir” is a term that refers to the anti-Black forms of misogyny that Black women experience. To explore how current automated hate speech detection approaches perform in detecting this type of hate, we evaluated the performance of two state-of-the-art detection tools, HateSonar and Google’s Perspective API, on a balanced dataset of 300 tweets, half of which are examples of misogynoir and half of which are examples of supporting Black women and an imbalanced dataset of 3138 tweets of which 162 tweets are examples of misogynoir and 2976 tweets are examples of allyship tweets. We aim to determine if these tools flag these messages under any of their classifications of hateful speech (e.g. “hate speech”, “offensive language”, “toxicity” etc.). Close analysis of the classifications and errors shows that current hate speech detection tools are ineffective in detecting misogynoir. They lack sensitivity to context, which is an essential component for misogynoir detection. We found that tweets likely to be classified as hate speech explicitly reference racism or sexism or use profane or aggressive words. Subtle tweets without references to these topics are more challenging to classify. We find that the lack of sensitivity to context may make such tools not only ineffective but potentially harmful to Black women.

show abstract

Section: Intersectional Hate Detectionmentioning

confidence: 99%

Misogynoir: challenges in detecting intersectional hate

Kwarteng

Perfumi

Farrell

et al. 2022

Soc. Netw. Anal. Min.

View full text Add to dashboard Cite

show abstract

“…When considering methods of mitigating bias in NLP, AAE's unique morphosyntactic structures are often neglected. Semantic context and lexical choice are more commonly accounted for (Barikeri et al, 2021;Cheng et al, 2022;Garimella et al, 2022;Hwang et al, 2020;Kiritchenko and Mohammad, 2018;Maronikolakis et al, 2022;Silva et al, 2021), but when focusing on improving a model's understanding of AAE, research often involves removing its morphological features (Tan et al, 2020) or translating between MAE and AAE (Ziems et al, 2023). In contrast, our work leverages AAE's morphosyntactic differences to improve disambiguation of habitual and non-habitual "be", rather than neutralizing the uniqueness of AAE.…”

Section: Related Workmentioning

confidence: 99%

Leveraging syntactic dependencies in disambiguation: the case of African American English

Previlon,

Rozet,

Gowda

et al. 2024

Preprint

View full text Add to dashboard Cite

African American English (AAE) has received recent attention in the field of natural language processing (NLP). Efforts to address bias against AAE in NLP systems tend to focus on lexical differences. Whenever the structural uniqueness of AAE is considered, the solution is often to remove or neutralize the differences. This work leverages knowledge about the unique morphosyntactic structures to improve automatic disambiguation of habitual and nonhabitual meanings of “be” in naturally produced AAE transcribed speech. Both meanings are employed in AAE but examples of Habitual be are rare in the already limited AAE data. Generally, representing contextual syntactic information improves semantic disambiguation of habituality. Using an ensemble of classical machine learning models with a representation of the unique POS and dependency patterns of Habitual be, we show that integrating syntactic information improves the identification of habitual uses of “be” by about 65 F1 points over a simple baseline model of n-grams, and as much as 74 points. The success of this approach demonstrates the potential impact when weembrace, rather than neutralize, the structural uniqueness of African American English.

show abstract

“…gender and race, intersect with and reinforce each other. Drawing on this framework, Tan and Celis (2019a); May et al (2019); Lepori (2020); Maronikolakis et al (2022); Guo and Caliskan (2021) analyse the compounding effects of race and gender encoded in contextualised word representations and downstream tasks. Recently, Lalor et al (2022);Jiang and Fellbaum (2020) show the harmful implications of intersectionality effects in pre-trained language models.…”

Section: Related Workmentioning

confidence: 99%

Measuring Intersectional Biases in Historical Documents

Borenstein,

Stanczak,

Rolskov

et al. 2023

Findings of the Association for Computational Linguistics: ACL 2023

View full text Add to dashboard Cite

Data-driven analyses of biases in historical texts can help illuminate the origin and development of biases prevailing in modern society. However, digitised historical documents pose a challenge for NLP practitioners as these corpora suffer from errors introduced by optical character recognition (OCR) and are written in an archaic language. In this paper, we investigate the continuities and transformations of bias in historical newspapers published in the Caribbean during the colonial era (18th to 19th centuries). Our analyses are performed along the axes of gender, race, and their intersection. We examine these biases by conducting a temporal study in which we measure the development of lexical associations using distributional semantics models and word embeddings. Further, we evaluate the effectiveness of techniques designed to process OCRgenerated data and assess their stability when trained on and applied to the noisy historical newspapers. We find that there is a trade-off between the stability of the word embeddings and their compatibility with the historical dataset. We provide evidence that gender and racial biases are interdependent, and their intersection triggers distinct effects. These findings align with the theory of intersectionality, which stresses that biases affecting people with multiple marginalised identities compound to more than the sum of their constituents. https://github.com/copenlu/i ntersectional-bias-pbw

show abstract

Analyzing Hate Speech Data along Racial, Gender and Intersectional Axes

Cited by 6 publications

References 15 publications

Misogynoir: challenges in detecting intersectional hate

Misogynoir: challenges in detecting intersectional hate

Leveraging syntactic dependencies in disambiguation: the case of African American English

Measuring Intersectional Biases in Historical Documents

Contact Info

Product

Resources

About