Ethical Challenges in Data-Driven Dialogue Systems

Henderson, Peter; Sinha, Koustuv; Angelard-Gontier, Nicolas; Ke, Nan Rosemary; Fried, G M; Lowe, Ryan; Pineau, Joëlle

doi:10.1145/3278721.3278777

Cited by 121 publications

(114 citation statements)

References 30 publications

Supporting

Mentioning

101

Contrasting

Unclassified

Order By: Relevance

“…Then, given a list of hot queries, a word embedding based function was used to find the similar queries with the responses. Henderson et al (2018) suggest that, due to their subjective nature and goal of mimicking human behaviour, datadriven dialogue models are susceptible to implicitly encode underlying biases in human dialogue, similar to related studies on biased lexical semantics derived from large corpora (Caliskan et al, 2017;Bolukbasi et al, 2016). By training a model on clean data, we aim to verify whether these models are able to provide more appropriate responses.…”

Section: Systems Evaluatedmentioning

confidence: 56%

“…This includes our own in-house bot which was trained on clean data. As such, the problem is not that the bot reflects bias in the data (Henderson et al, 2018), but how humans construct contextual meaning. 22 Some (of the less offensive) examples include: Prompt: "I love watching porn."…”

Section: Prompt Contextmentioning

confidence: 99%

“…In contrast to previous work, we also include current data-driven systems in our study. We explore the hypothesis that unethical system behaviour might be caused by biased data sets (Henderson et al, 2018), by training our own sequenceto-sequence model (Seq2Seq) (Sutskever et al, 2014) on "clean" data. We ground our response stimuli in (anonymised) customer data gathered during the Amazon Alexa Challenge 2017.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Proceedings of the Second ACL Workshop on Ethics in Natural Language Processing

2018

View full text Add to dashboard Cite

His research interests lie at the intersection of philosophy of mind and action, metaphysics, and ethics. He has published on free will and the impact of machine learning on ethical decisions. Katherine Bailey (Acquia) is a researcher and team leader in industry. Her recent work has been on machine learning applications for natural language processing and other fields. She is pioneering a "fewshot learning" approach which promises greater efficiency in machine learning. Katherine has spoken at international conferences on both the technical details of artificial intelligence and the ethical issues that arise from its use in a variety of contexts. Francien Dechesne (Leiden University, The Netherlands), is a researcher at the Center for Law and Digital Technologies (eLaw) of the Leiden Law School, and lecturer at TU Eindhoven. Her research lies at the intersection between societal and ethical issues of information and communication technologies, including the question of how to balance public, commercial, and individual interests in data-driven innovations. In particular, her research focuses on potential negative societal impact of decisions based on data-analytics, and the design of accountability mechanisms to address this impact. We received fewer paper submissions than in the previous year, but present a large range of topics, addressing issues related to overgeneralization, dual use, privacy protection, bias in NLP models, underrepresentation, fairness, and more. Authors share insights about the intersection of NLP and ethics in academic, industrial, and clinical work. We selected three papers for oral presentation. Due to the involvement of different disciplines with differing publication traditions, we also offered a non-archival submission option, which means not all papers presented at the workshop are included here. We are glad to see the continued interest in this important topic and hope that this workshop will help defining ethical issues in NLP, and raising awareness of ethical considerations throughout the community.

show abstract

Section: Systems Evaluatedmentioning

confidence: 56%

Section: Prompt Contextmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Proceedings of the Second ACL Workshop on Ethics in Natural Language Processing

2018

View full text Add to dashboard Cite

show abstract

“…To characterize the types of paraphrasing issues, two authors of this paper investigated the crowd-sourced paraphrases, and recognized 5 primary categories of paraphrasing issues. However, we only considered paraphrase-level issues related to the validity of a paraphrase without considering dataset-level quality issues such as lexical diversity (Negri et al, 2012) and bias (Henderson et al, 2018).…”

Section: Common Paraphrasing Issuesmentioning

confidence: 99%

“…A lack of variations in training samples can result in incorrect intent detection and consequently execution of undesirable tasks (e.g., booking an expensive hotel instead of a cheap room) (Henderson et al, 2018). Likewise, quality issues in the training samples can lead to unmitigated disasters (Neff and Nagy, 2016) as it happened to Microsoft's Tay by making a huge number of offensive commentaries due to biases in the training data (Henderson et al, 2018). It is therefore not surprising that research and development into training data acquisition for bots has received significant consideration (Campagna et al, 2017;Kang et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

A Study of Incorrect Paraphrases in Crowdsourced User Utterances

Yaghoub-Zadeh-Fard

Benatallah

Barukh

et al. 2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

Developing bots demands high quality training samples, typically in the form of user utterances and their associated intents. Given the fuzzy nature of human language, such datasets ideally must cover all possible utterances of each single intent. Crowdsourcing has widely been used to collect such inclusive datasets by paraphrasing an initial utterance. However, the quality of this approach often suffers from various issues, particularly language errors produced by unqualified crowd workers. More so, since workers are tasked to write open-ended text, it is very challenging to automatically asses the quality of paraphrased utterances. In this paper, we investigate common crowdsourced paraphrasing issues, and propose an annotated dataset called Para-Quality, for detecting the quality issues. We also investigate existing tools and services to provide baselines for detecting each category of issues. In all, this work presents a data-driven view of incorrect paraphrases during the bot development process, and we pave the way towards automatic detection of unqualified paraphrases.

show abstract

On safe and usable chatbots for promoting voter participation

Muppasani,

Pallagani,

Lakkaraju

et al. 2023

AI Magazine

View full text Add to dashboard Cite

Chatbots, or bots for short, are multimodal collaborative assistants that can help people complete useful tasks. Usually, when chatbots are referenced in connection with elections, they often draw negative reactions due to the fear of mis‐information and hacking. Instead, in this work, we explore how chatbots may be used to promote voter participation in vulnerable segments of society like senior citizens and first‐time voters. In particular, we have built a system that amplifies official information while personalizing it to users' unique needs transparently (e.g., language, cognitive abilities, linguistic abilities). The uniqueness of this work are (a) a safe design where only responses that are grounded and traceable to an allowed source (e.g., official question/answer) will be answered via system's self‐awareness (metacognition), (b) a do‐not‐respond strategy that can handle customizable responses/deflection, and (c) a low‐programming design‐pattern based on the open‐source Rasa platform to generate chatbots quickly for any region. Our current prototypes use frequently asked questions (FAQ) election information for two US states that are low on an ease‐of‐voting scale, and have performed initial evaluations using focus groups with senior citizens. Our approach can be a win‐win for voters, election agencies trying to fulfill their mandate and democracy at large.

show abstract

Ethical Challenges in Data-Driven Dialogue Systems

Cited by 121 publications

References 30 publications

Proceedings of the Second ACL Workshop on Ethics in Natural Language Processing

Proceedings of the Second ACL Workshop on Ethics in Natural Language Processing

A Study of Incorrect Paraphrases in Crowdsourced User Utterances

On safe and usable chatbots for promoting voter participation

Contact Info

Product

Resources

About