Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semantics

Abstract

Word embeddings are numeric representations of meaning derived from word co-occurrence statistics in corpora of human-produced texts. The statistical regularities in language corpora encode wellknown social biases into word embeddings (e.g., the word vector for family is closer to the vector women than to men). Although efforts have been made to mitigate bias in word embeddings, with the hope of improving fairness in downstream Natural Language Processing (NLP) applications, these efforts will remain limited until we more deeply understand the multiple (and often subtle) ways that social biases can be reflected in word embeddings. Here, we focus on gender to provide a comprehensive analysis of group-based biases in widely-used static English word embeddings trained on internet corpora (GloVe 2014, fastText 2017). While some previous research has helped uncover biases in specific semantic associations between a group and a target domain (e.g., women -family), using the Single-Category Word Embedding Association Test, we demonstrate the widespread prevalence of gender biases that also show differences in: (1) frequencies of words associated with men versus women; (b) part-of-speech tags in gender-associated words; (c) semantic categories in gender-associated words; and (d) valence, arousal, and dominance in gender-associated words. We leave the analysis of non-binary gender to future work due to the challenges in accurate group representation caused by limitations inherent in data.First, in terms of word frequency: we find that, of the 1,000 most frequent words in the vocabulary, 77% are more associated with men than women, providing direct evidence of a masculine default in the everyday language of the English-speaking world. Second, turning to parts-of-speech: the top male-associated words are typically verbs (e.g., fight, overpower) while the top female-associated words are typically adjectives and adverbs (e.g., giving, emotionally). Gender biases in embeddings also permeate parts-of-speech. Third, This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.

Exploring explainable AI: category theory insights into machine learning algorithms

“…One of the important ideas that follow from the Yoneda embedding is the idea of a representable functor. This is a functor between categories F : C → D such that there exists an object c of the category C satisfying F(x) = h c (x) = hom(c, x) for all objects x of C. In the case of word embeddings, if one wishes to know if a certain embedded word c is gender biased one would look at the difference between the cosine distance between the word c and the words man and woman (see [CAC+22]). This can be translated, in the enriched setting, into h c (man) − h c (woman) .…”