Guiding Principles for Participatory Design-inspired Natural Language Processing

Abstract

We introduce 9 guiding principles 1 to integrate Participatory Design (PD) methods in the development of Natural Language Processing (NLP) systems. The adoption of PD methods by NLP will help to alleviate issues concerning the development of more democratic, fairer, less-biased technologies to process natural language data. This short paper is the outcome of an ongoing dialogue between designers and NLP experts and adopts a non-standard format following previous work by Traum (2000); Bender (2013); Abzianidze and Bos (2019). Every section is a guiding principle. While principles 1-3 illustrate assumptions and methods that inform community-based PD practices, we used two fictional design scenarios (Encinas and Blythe, 2018), which build on top of situations familiar to the authors, to elicit the identification of the other 6. Principles 4-6 describes the impact of PD methods on the design of NLP systems, targeting two critical aspects: data collection & annotation, and the deployment & evaluation. Finally, principles 7-9 guide a new reflexivity of the NLP research with respect to its context, actors and participants, and aims. We hope this guide will offer inspiration and a road-map to develop a new generation of PD-inspired NLP.

Coimagining the Future of Voice Assistants with Cultural Sensitivity

“…In NLP [23,36,37], co-design centres around crowdsourcing: gathering large numbers of people who complete microtasks towards a common goal. Crowdsourcing has been used to gather utterances and dialogues within particular contexts of use [15,36,38], translate and localize existing data sets [39,40], and elicit reactions to voice stimuli, especially social and paralinguistic characteristics [41,42].…”

“I'm” Lost in Translation: Pronoun Missteps in Crowdsourced Data Sets

“…We used the pronoun dictionary provided by the nltk package's Part of Speech (POS) 11 library for the English data set, and the tokenizer created by Kudo, Yamamoto, and Matsumoto [24] for the Japanese data set. However, because these libraries did not represent an up-to-date or inclusive spread of pronouns, we manually added several pronouns according to recent NLP work [25].…”

“…Initiatives in natural language processing (NLP) for communication between people and virtual assistants (VAs) is taking off [9,28,36]. Many are gathering and offering large, crowdsourced data sets for this purpose [11]. An ongoing challenge is that most are in English and geared towards the US context [12,20,23].…”

A Multilingual Approach to Identify and Classify Exceptional Measures against COVID-19

“…The reliance on multi-lingual methods adopts a philosophical perspective of Artificial Intelligence (AI) as a problem-solving tool rather than as an adaptive mechanism mimicking human abilities (Winograd, 1997;Auernhammer, 2020;Caselli et al, 2021). The "intelligent" systems developed in this project (see Section 5) do not aim at substituting humans but are designed to account for different development cycles with humans in the loop.…”