“I'm” Lost in Translation: Pronoun Missteps in Crowdsourced Data Sets

Abstract

As virtual assistants continue to be taken up globally, there is an ever-greater need for these speech-based systems to communicate naturally in a variety of languages. Crowdsourcing initiatives have focused on multilingual translation of big, open data sets for use in natural language processing (NLP). Yet, language translation is often not one-to-one, and biases can trickle in. In this late-breaking work, we focus on the case of pronouns translated between English and Japanese in the crowdsourced Tatoeba database. We found that masculine pronoun biases were present overall, even though plurality in language was accounted for in other ways. Importantly, we detected biases in the translation process that reflect nuanced reactions to the presence of feminine, neutral, and/or non-binary pronouns. We raise the issue of translation bias for pronouns and offer a practical solution to embed plurality in NLP data sets.

Coimagining the Future of Voice Assistants with Cultural Sensitivity

“…Notably, translations of English and Japanese NLP data sets are common, such as the crowdsourcing initiatives of Tatoeba (https://tatoeba .org/en/downloads) and MASSIVE [27]. Yet, biases have been found within these data sets: "missteps" resulting from the crowdsourced translation process [55]. This suggests that translation may be insufficient.…”