Delphi: Towards Machine Ethics and Norms

Jiang, Liwei; Hwang, Jena D.; Bhagavatula, Chandra; Bras, Ronan Le; Liang, Jing; Dodge, Jesse; Sakaguchi, Kei; Forbes, Maxwell; Borchardt, Jon; Gabriel, Saadia; Tsvetkov, Yulia; Etzioni, Oren; Sap, Maarten; Rini, Regina; Choi, Yejin

doi:10.48550/arxiv.2110.07574

Cited by 19 publications

(14 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results were less than encouraging, with Nobles et al [72] finding that the majority of the time, chatbots responded in inappropriate ways in these situations, ranging from simply saying "can you repeat that" to giving actively harmful information. Another related line of work is context-aware, long-form, ethical and persona-based, response generation [62,60,43,55], where the chatbot or dialog system is supposed to hold a conversation with previous context taken into consideration [50]. This context could be the persona of the user, or previous conversations.…”

Section: Secrets Are Contextualmentioning

confidence: 99%

What Does it Mean for a Language Model to Preserve Privacy?

Brown¹,

Lee²,

Mireshghallah³

et al. 2022

Preprint

View full text Add to dashboard Cite

Natural language reflects our private lives and identities, making its privacy concerns as broad as those of real life. Language models lack the ability to understand the context and sensitivity of text, and tend to memorize phrases present in their training sets. An adversary can exploit this tendency to extract training data. Depending on the nature of the content and the context in which this data was collected, this could violate expectations of privacy. Thus, there is a growing interest in techniques for training language models that preserve privacy. In this paper, we discuss the mismatch between the narrow assumptions made by popular data protection techniques (data sanitization and differential privacy), and the broadness of natural language and of privacy as a social norm. We argue that existing protection methods cannot guarantee a generic and meaningful notion of privacy for language models. We conclude that language models should be trained on text data which was explicitly produced for public use.

show abstract

Section: Secrets Are Contextualmentioning

confidence: 99%

What Does it Mean for a Language Model to Preserve Privacy?

Brown¹,

Lee²,

Mireshghallah³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent progress in foundational language models like GPT-n, BERT, ELMo, and others combined with the crowdsourced datasets containing text snippets on social and ethical norms allowed researchers to build AI systems that are particularly fine-tuned for tasks in moral decision-making. These systems are supposed to 'facilitate […] ethical interactions between AI systems and humans' [132]. Hence, one would expect that especially morally informed AI systems are particularly sensitive to biases or discrimination and possess high ethical standards due to their exclusive exposure to training stimuli that represent ethical judgments [133][134][135][136][137].…”

Section: A Cow Stands Next To a Camelmentioning

confidence: 99%

Speciesist bias in AI: how AI applications perpetuate discrimination and unfair outcomes against animals

Hagendorff

Bossert²,

Tse

et al. 2022

AI Ethics

View full text Add to dashboard Cite

Massive efforts are made to reduce biases in both data and algorithms to render AI applications fair. These efforts are propelled by various high-profile cases where biased algorithmic decision-making caused harm to women, people of color, minorities, etc. However, the AI fairness field still succumbs to a blind spot, namely its insensitivity to discrimination against animals. This paper is a critical comment on current fairness research in AI. It is the first to describe the ‘speciesist bias’ and investigate it in several different AI systems by reflecting on the problem via a normative analysis and by probing, in several case studies, image recognition, word embedding, and language models with established methods for bias detection. We claim that animals matter morally and that discriminating against them is unethical. Furthermore, we provide evidence for speciesist biases in all the mentioned areas of AI. We find that speciesist biases are solidified by many mainstream AI applications, especially in the fields of computer vision as well as natural language processing. In both cases, this occurs because the models are trained on datasets in which speciesist patterns prevail. Therefore, AI technologies currently play a significant role in perpetuating and normalizing violence against animals. To change this, AI fairness frameworks must widen their scope and include mitigation measures for speciesist biases. This paper addresses the AI community in this regard and stresses the influence AI systems can have on either increasing or reducing the violence that is inflicted on animals, especially on farmed animals.

show abstract

“…[1502] also proposes a bot-adversarial dialogue framework to collect unsafe samples in conversational testing, which would be modified and used to re-train conversational models as "safety layer". Dialogue systems integrated multiple safety improvements are proved to have stronger reliability [1522,430].…”

Section: Safety and Ethical Riskmentioning

confidence: 99%

A Roadmap for Big Model

Yuan¹,

Zhao²,

Jiahong³

et al. 2022

Preprint

View full text Add to dashboard Cite

domains indexed by Google News. It contains 31 million documents with an average length of 793 BPE tokens. Like C4, it excludes examples with duplicate URLs. News dumps from December 2016 through March 2019 were used as training data, articles published in April 2019 from the April 2019 dump were used for evaluation. OpenWebText2(OWT2). OWT2 is an enhanced version of the original OpenWebTextCorpus, including content from multiple languages, document metadata, multiple dataset versions, and open source replication code, covering all Reddit submissions from 2005 up until April 2020. PubMed Central(PMC). PMC is a free full-text archive of biomedical and life sciences journal literature from the U.S. National Institutes of Health's National Library of Medicine (NIH/NLM). The dataset is updated daily. In addition to full-text articles, they contain corrections, retractions, and expressions of concern, as well as file lists that include metadata for articles in each dataset.PMC obtained by open registration in Amazon Web Services (AWS) includes The PMC Open Access Subset and The Author Manuscript Dataset. The PMC Open Access Subset includes all articles and preprints in PMC with a machine-readable Creative Commons license that allows reuse. The Author Manuscript Dataset includes accepted author manuscripts collected under a funder policy in PMC and made available in machine-readable formats for text mining. ArXiv. ArXiv is a repository of 1.7 million articles, with relevant features such as article titles, authors, categories, abstracts, full text PDFs, and more. It provides open access to academic articles, covering many subdisciplines from vast branches of physics to computer science to everything in between, including math, statistics, electrical engineering, quantitative biology, and economics, which is helpful to the potential downstream applications of the research field. In addition, the writing language of LaTeX also contributes to the study of language models. Colossal Clean Crawled Corpus(C4). C4 is a colossal, cleaned version of Common Crawl's web crawl corpus. It is based on Common Crawl dataset and was used to train the T5 text-to-text Transformer models. The cleaned English version of C4 has 364,868,901 training examples and 364,608 validation examples, while the uncleaned English version has 1,063,805,324 training examples and 1,065,029 validation examples; the realnewslike version has 13,799,838 training examples and 13,863 validation examples, while the webtextlike version has 4,500,788 training examples and 4,493 validation examples. Wiki-40B. Wikipedia (Wiki-40B) is a clean-up text collection containing more than 40 Wikipedia language editions of pages corresponding to entities. The dataset is split into train/validation/test sets for each language. The training set has 2,926,536 examples, the validation set has 163,597 examples, and the test set has 162,274 examples. Wiki-40B is cleaned by a page filter to remove ambiguous, redirected, deleted, and non-physical pages. CLUECorpus2020. CLUECorpus2020 ...

show abstract

Delphi: Towards Machine Ethics and Norms

Cited by 19 publications

References 24 publications

What Does it Mean for a Language Model to Preserve Privacy?

What Does it Mean for a Language Model to Preserve Privacy?

Speciesist bias in AI: how AI applications perpetuate discrimination and unfair outcomes against animals

A Roadmap for Big Model

Contact Info

Product

Resources

About