“I’m fully who I am”: Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation

Ovalle, Anaelia; Goyal, Palash; Dhamala, Jwala; Jaggers, Zachary; Chang, Kai-Wei; Galstyan, Aram; Zemel, Richard S.; Gupta, Rahul

doi:10.1145/3593013.3594078

Cited by 13 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notably, a substantial set of search results focuses on bias and discrimination. These are most typically related to LLMs, and include works such as van der Wal et al [227] and Huang et al [99], which address multiple biases and harmful stereotypes, and others, like Abid et al [1], Felkner et al [72], Ovalle et al [159], and Gadiraju et al [76] that concentrate on specific societal biases. The result set also reveals that papers that focus on specific or a limited number of risks and harms often scrutinize issues of unreliable performance of foundation models, or misinformation and propaganda.…”

Section: Mapping Individual Social and Biospheric Impacts Of Foundati...mentioning

confidence: 99%

Mapping the individual, social and biospheric impacts of Foundation Models

Domínguez Hernández,

Krishna,

Perini

et al. 2024

The 2024 ACM Conference on Fairness, Accountability, and Transparency

View full text Add to dashboard Cite

Responding to the rapid roll-out and large-scale commercialization of foundation models, large language models, and generative AI, an emerging body of work is shedding light on the myriad impacts these technologies are having across society. Such research is expansive, ranging from the production of discriminatory, fake and toxic outputs, and privacy and copyright violations, to the unjust extraction of labor and natural resources. The same has not been the case in some of the most prominent AI governance initiatives in the global north like the UK's AI Safety Summit and the G7's Hiroshima process, which have influenced much of the international dialogue around AI governance. Despite the wealth of cautionary tales and evidence of algorithmic harm, there has been an ongoing over-emphasis within the AI governance discourse on technical matters of safety and global catastrophic or existential risks. This narrowed focus has tended to draw attention away from very pressing social and ethical challenges posed by the current brute-force industrialization of AI applications. To address such a visibility gap between real-world consequences and speculative risks, this paper offers a critical framework to account for the social, political, and environmental dimensions of foundation models and generative AI. Drawing on a review of the literature on the harms and risks of * Equal contribution as lead authors † Also with The Alan Turing Institute.

show abstract

Section: Mapping Individual Social and Biospheric Impacts Of Foundati...mentioning

confidence: 99%

Mapping the individual, social and biospheric impacts of Foundation Models

Domínguez Hernández,

Krishna,

Perini

et al. 2024

The 2024 ACM Conference on Fairness, Accountability, and Transparency

View full text Add to dashboard Cite

show abstract

“…The last few years have witnessed and increasing attention toward (binary) gender bias in NLP (Sun et al, 2019;Stanczak and Augenstein, 2021;Savoldi et al, 2022a). Concurrently, emerging research has highlighted the importance of reshaping gender in NLP technologies in a more inclusive manner (Dev et al, 2021), also through the representation of non-binary identities in language (Lauscher et al, 2022;Ovalle et al, 2023). Foundational works in this area have included several applications, such as coreference resolution systems (Cao and Brandl et al, 2022) and fair rewriters (Vanmassenhove et al, 2021;Amrhein et al, 2023).…”

Section: Related Workmentioning

confidence: 99%

On the Dynamics of Gender Learning in Speech Translation

Savoldi¹,

Gaido²,

Bentivogli³

et al. 2022

Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP)

View full text Add to dashboard Cite

Due to the complexity of bias and the opaque nature of current neural approaches, there is a rising interest in auditing language technologies. In this work, we contribute to such a line of inquiry by exploring the emergence of gender bias in Speech Translation (ST). As a new perspective, rather than focusing on the final systems only, we examine their evolution over the course of training. In this way, we are able to account for different variables related to the learning dynamics of gender translation, and investigate when and how gender divides emerge in ST. Accordingly, for three language pairs (en -> es, fr, it) we compare how ST systems behave for masculine and feminine translation at several levels of granularity. We find that masculine and feminine curves are dissimilar, with the feminine one being characterized by a more erratic behaviour and late improvements over the course of training. Also, depending on the considered phenomena, their learning trends can be either antiphase or parallel. Overall, we show how such a progressive analysis can inform on the reliability and time-wise acquisition of gender, which is concealed by static evaluations and standard metrics.

show abstract

“…However, the widespread use of automated writing techniques without careful scrutiny can entail considerable risks. Recent studies have shown that Natural Language Generation (NLG) models are gender biased (Sheng et al, 2019(Sheng et al, , 2020Sheng et al, 2021a;Bender et al, 2021) and therefore pose a risk to harm minorities when used in sensitive applications (Sheng et al, 2021b;Ovalle et al, 2023a;Prates et al, 2018). Such biases might also infiltrate the application of automated reference letter generation and cause substantial societal harm, as research in social sciences (Madera et al, 2009;Khan et al, 2021) unveiled how biases in professional documents lead to diminished career opportunities for gender minority groups.…”

Section: Introductionmentioning

confidence: 99%

“Kelly is a Warm Person, Joseph is a Role Model”: Gender Biases in LLM-Generated Reference Letters

Wan,

Pu,

Sun

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Large Language Models (LLMs) have recently emerged as an effective tool to assist individuals in writing various types of content, including professional documents such as recommendation letters. Though bringing convenience, this application also introduces unprecedented fairness concerns. Model-generated reference letters might be directly used by users in professional scenarios. If underlying biases exist in these model-constructed letters, using them without scrutinization could lead to direct societal harms, such as sabotaging application success rates for female applicants. In light of this pressing issue, it is imminent and necessary to comprehensively study fairness issues and associated harms in this real-world use case. In this paper, we critically examine gender biases in LLM-generated reference letters. Drawing inspiration from social science findings, we design evaluation methods to manifest biases through 2 dimensions: (1) biases in language style and (2) biases in lexical content. We further investigate the extent of bias propagation by analyzing the hallucination bias of models, a term that we define to be bias exacerbation in model-hallucinated contents. Through benchmarking evaluation on 2 popular LLMs-Chat-GPT and Alpaca, we reveal significant gender biases in LLM-generated recommendation letters. Our findings not only warn against using LLMs for this application without scrutinization, but also illuminate the importance of thoroughly studying hidden biases and harms in LLM-generated professional documents.

show abstract

“I’m fully who I am”: Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation

Cited by 13 publications

References 19 publications

Mapping the individual, social and biospheric impacts of Foundation Models

Mapping the individual, social and biospheric impacts of Foundation Models

On the Dynamics of Gender Learning in Speech Translation

“Kelly is a Warm Person, Joseph is a Role Model”: Gender Biases in LLM-Generated Reference Letters

Contact Info

Product

Resources

About