Challenges in Detoxifying Language Models

Welbl, Johannes; Glaese, Amelia; Uesato, Jonathan; Dathathri, Sumanth; Mellor, John; Hendricks, Lisa Anne; Anderson, Kirsty; Kohli, Pushmeet; Coppin, Ben; Huang, Po-Sen

doi:10.48550/arxiv.2109.07445

Cited by 18 publications

(38 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Source: https://bit.ly/3zBLnry. On the other hand, as recently demonstrated in studies such as [60,61], granular safe filtering of the datasets created and the downstream detoxification of the models trained on such datasets remain a tenuous and laborious work. When one juxtaposes the financial compensation levels and investments that went into the teams that have undertaken these detoxification challenges, the asymmetry becomes even more stark.…”

Section: 1 Asymmetry Of Efforts: Crawling V/s Detoxificationmentioning

confidence: 99%

Multimodal datasets: misogyny, pornography, and malignant stereotypes

Birhane,

Prabhu,

Kahembwe

2021

Preprint

View full text Add to dashboard Cite

We have now entered the era of trillion parameter machine learning models trained on billion-sized datasets scraped from the internet. The rise of these gargantuan datasets has given rise to formidable bodies of critical work that has called for caution while generating these large datasets. These address concerns surrounding the dubious curation practices used to generate these datasets, the sordid quality of alt-text data available on the world wide web, the problematic content of the CommonCrawl dataset often used as a source for training large language models, and the entrenched biases in large-scale visio-linguistic models (such as OpenAI's CLIP model) trained on opaque datasets (WebImageText). In the backdrop of these specific calls of caution, we examine the recently released LAION-400M dataset, which is a CLIP-filtered dataset of Image-Alt-text pairs parsed from the Common-Crawl dataset. We found that the dataset contains, troublesome and explicit images and text pairs of rape, pornography, malign stereotypes, racist and ethnic slurs, and other extremely problematic content. We outline numerous implications, concerns and downstream harms regarding the current state of large scale datasets while raising open questions for various stakeholders including the AI community, regulators, policy makers and data subjects. Warning: This paper contains NSFW content that some readers may find disturbing, distressing, and/or offensive.

show abstract

Section: 1 Asymmetry Of Efforts: Crawling V/s Detoxificationmentioning

confidence: 99%

Multimodal datasets: misogyny, pornography, and malignant stereotypes

Birhane,

Prabhu,

Kahembwe

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Asian Chinese (23) , slim (29) , yellow (39) , Japanese (50) , average (55) , straight (70) , inscrutable (72) , desirable (77) , feminine (88) , pleasant (91) Black civil (29) , lazy (44) , immoral (53) , animalistic (54) , capable (66) , equal (73) , stupid (74) , lower (78) , athletic (88) , incapable (82) White fair (62) , true (68) , ultimate (71) , higher (72) , virtuous (74) , racist (79) , non-white (82) , civilized (83) , pale (90) , responsible (92)…”

Section: Male Identifiersmentioning

confidence: 99%

“…Gender Analysis Encouragingly, we note that, for gender, among the top 100 most frequent adjectives, almost 80 were exactly the same, as shown in Figure 3. In the figure, words are ordered left-to-right in order Male top (51) , violent (53) , eccentric (59) , military (60) , polite (62) , serious (63) , national (67) , different (68) , aggressive (71) , right (78) Female beautiful (2) , attractive (37) , female (45) , mental (50) , sweet (57) , charitable (60) , perfect (62) , slim (67) , only (72) , excited (74) Table 8: Top 10 distinct words with the highest frequency from the 100 most frequent words that occurred for Male and Female identifiers. The numbers in parenthesis represent the word's ordinal position in the top 100 most frequent words list.…”

Section: Male Identifiersmentioning

confidence: 99%

“…Training set filtering -where the elements of the training dataset are analyzed and elements that show evidence of bias are removed from the training data [44]. b) Training set modification -where elements of the training dataset are randomized with respect tovariables such as gender and ethnicity that should be neutral with respect to the subject matter[72].c) Prompt engineering -where the inputs to the model for each query are modified to steer the model away from bias[60,14].d) Fine tuning -where the trained model is retrained to unlearn biased tendencies[19,20,28].e) Output steering -where a filtering step is added to the inference procedure to re-weight output values and steer the output away from biased responses.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Smith¹,

Patwary²,

Norick³

et al. 2022

Preprint

125

133

View full text Add to dashboard Cite

Pretrained general-purpose language models can achieve state-of-the-art accuracies in various natural language processing domains by adapting to downstream tasks via zero-shot, few-shot and finetuning techniques. Because of their success, the size of these models has increased rapidly, requiring high-performance hardware, software, and algorithmic techniques to enable training such large models. As the result of a joint effort between Microsoft and NVIDIA, we present details on the training of the largest monolithic transformer based language model, Megatron-Turing NLG 530B (MT-NLG), with 530 billion parameters. In this paper, we first focus on the infrastructure as well as the 3D parallelism methodology used to train this model using DeepSpeed and Megatron. Next, we detail the training process, the design of our training corpus, and our data curation techniques, which we believe is a key ingredient to the success of the model. Finally, we discuss various evaluation results, as well as other interesting observations and new properties exhibited by MT-NLG. We demonstrate that MT-NLG achieves superior zero-, one-, and few-shot learning accuracies on several NLP benchmarks and establishes new state-of-the-art results. We believe that our contributions will help further the development of large-scale training infrastructures, large-scale language models, and natural language generations.

show abstract

“…Experiments conducted on GPT-3 (which has been trained on 570GB of text data from Common Crawl) show that the model may generate toxic sentences even when prompted with non-toxic text [11]. Although applying filtering of training data using automated toxicity scores may introduce classifier-specific biases [12], this technique remains more effective than decoder-based detoxification using methods such as swear word filters, PPLM [13], soft prompt tuning [14] or toxicity control tokens [15].…”

Section: Introductionmentioning

confidence: 99%

Cedille: A large autoregressive French language model

Müller¹,

Laurent²

2022

Preprint

View full text Add to dashboard Cite

Scaling up the size and training of autoregressive language models has enabled novel ways of solving Natural Language Processing tasks using zero-shot and few-shot learning. While extreme-scale language models such as GPT-3 offer multilingual capabilities, zero-shot learning for languages other than English remain largely unexplored. Here, we introduce Cedille, a large open source auto-regressive language model, specifically trained for the French language. Our results show that Cedille outperforms existing French language models and is competitive with GPT-3 on a range of French zero-shot benchmarks. Furthermore, we provide an in-depth comparison of the toxicity exhibited by these models, showing that Cedille marks an improvement in language model safety thanks to dataset filtering.

show abstract

Challenges in Detoxifying Language Models

Cited by 18 publications

References 45 publications

Multimodal datasets: misogyny, pornography, and malignant stereotypes

Multimodal datasets: misogyny, pornography, and malignant stereotypes

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Cedille: A large autoregressive French language model

Contact Info

Product

Resources

About