Analyzing Stereotypes in Generative Text Inference Tasks

Abstract

Stereotypes are inferences drawn about people based on their demographic attributes, which may result in harms to users when a system is deployed. In generative language-inference tasks, given a premise, a model produces plausible hypotheses that follow either logically (natural language inference) or commonsensically (commonsense inference). Such tasks are therefore a fruitful setting in which to explore the degree to which NLP systems encode stereotypes. In our work, we study how stereotypes manifest when the potential targets of stereotypes are situated in real-life, neutral contexts. We collect human judgments on the presence of stereotypes in generated inferences, and compare how perceptions of stereotypes vary due to annotator positionality. DomainTarget Categories Gender man, woman, non-binary person, trans man, trans woman, cis man, cis woman

Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models

“…Table 3 lists all the individual social groups we cover in this work. We manually construct the list by combining and picking groups from the list of social groups from Sotnikova et al (2021) and Koch et al (2016) and also adding social groups we think are stereotyped in U.S. culture.…”

Exploring Social Biases of Large Language Models in a College Artificial Intelligence Course

“…Much existing work focuses on diagnosing representational harms with bias probe tasks: tasks that measure whether a model's predictions differ between two (or more) groups of interest. A number of probe tasks have been proposed: Rudinger, May, and Van Durme (2017); Sheng et al (2019); Bordia and Bowman (2019); Lee, Madotto, and Fung (2019); Liu et al (2019a);May et al (2019); Nadeem, Bethke, and Reddy (2021); Sotnikova et al (2021) and others. Most of these focus on gender stereotypes.…”