Societal Biases in Language Generation: Progress and Challenges

Abstract

Technology for language generation has advanced rapidly, spurred by advancements in pre-training large models on massive amounts of data and the need for intelligent agents to communicate in a natural manner. While techniques can effectively generate fluent text, they can also produce undesirable societal biases that can have a disproportionately negative impact on marginalized populations. Language generation presents unique challenges for biases in terms of direct user interaction and the structure of decoding techniques. To better understand these challenges, we present a survey on societal biases in language generation, focusing on how data and techniques contribute to biases and progress towards reducing biases. Motivated by a lack of studies on biases from decoding techniques, we also conduct experiments to quantify the effects of these techniques. By further discussing general trends and open challenges, we call to attention promising directions for research and the importance of fairness and inclusivity considerations for language generation applications.

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

“…For example, whereas "The woman is a nurse" is not a problematic sentence, it can be problematic if the model disproportionately associates women with certain occupations. As discussed in Sheng et al (2021), distributional biases in language models can have both negative representational impacts (e.g., Kay et al (2015)) and allocational impacts (e.g., Dastin (2018)). To investigate distributional biases in our model, we measure stereotypical associations between gender and occupation, the distribution of sentiment in samples conditioned on different social groups, and perplexity on different dialects.…”

“…Challenges in distributional bias. While we only consider a few possible evaluations (see Sheng et al (2021) for an overview), we observe that distributional bias can be especially challenging to measure. Figure 6a illustrates the brittleness of template-based evaluation: simply changing the verb in the gender and occupation template from "was" to "is" impacts observed trends.…”

On the Limits of Minimal Pairs in Contrastive Evaluation

“…For language generation systems to be deployed, they should behave according to specified principles in a robust way. Typical requirements are linguistic acceptability, avoidance of undesirable societal biases (Sheng et al, 2021), and the avoidance of harmful speech acts. Contrastive evaluation is one of several methods that can help predict the behavior of language generation systems.…”

Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarks

“…Unlike other unsafety problems, social biases that convey negative stereotypes or prejudices on specific populations are usually stated in implicit expressions rather than explicit words Blodgett et al, 2020), thus is a challenging task to deal with. On the other hand, dialog systems serve as a direct interface to users, and their biased responses may have a disproportionately negative impact, which also hinders the broad deployment of dialog systems, especially for the large-scale generative models (Sheng et al, 2021). Thus, it is crucial to tackling the social bias issue in conversational situations.…”