The Eliza effect and its dangers: from demystification to gender critique

Abstract

This essay provides a gender critique of the Eliza effect. It delineates the way in which the Eliza effect is operationalised in AI research even as it is ostensibly demystified, for example in the writings of Douglas Hofstadter and Joseph Weizenbaum. It then exposes the gendered assumptions embedded in the nomenclature used to name this misperception of the computer as having capabilities equivalent to the human. It traces the genealogy of that nomenclature back through Weizenbaum's ELIZA, to George Bernard Shaw's Pygmalion. A close reading of the play is deployed in order to reveal the structural inequities of gender, class, and who or what gets to be human, that are both explored in the play and encoded in the operation and operationalisations of the Eliza effect. It concludes by attending to that operation and operationalisation in relation to today's Virtual Personal Assistant's, and makes a case for the importance of critique in order to expose the inequitable structures of power obscured and compounded by the Eliza effect -both its name, and that which it names.

Pygmalion Displacement: When Humanising AI Dehumanises Women

“…Pedro was operated solely by women, despite the fact the name adopted is stereotypically male. The first modern chatbot, however, is often considered to be ELIZA, created by Joseph Weizenbaum in 1964 to simulate a therapist that resulted in users believing a real person was behind the automated responses (Dillon, 2020;Hirshbein, 2004). The mechanism behind ELIZA was simple pattern-matching, but it managed to fool people enough to be considered to have passed the Turing test.…”

“…Pedro and ELIZA were succeeded by more modern digital assistants, including both chatbots and voice assistants. Dillon (2020) notes that, generally speaking, "whereas digital assistants are gendered as female, digital advisors (operating in legal, financial or medical contexts, for instance) are gendered as male" (references removed, emphasis in original). Here, we focus on digital assistants (and not advisors), looking through the Pygmalion lens (Table 1) to see how these are caught up in processes of Pygmalion displacement.…”

The Effects of AI on the Working Lives of Women

“…Bergen (2016) notes that VPAs "rely on the audible performance of gender that capitalises on associations between the feminine and affective labour" (Bergen, 2016). Dillon (2020) argues that "[f]emale VPAs transfer to the digital realm the gendering and stratification of labour found in the real world with working women principally confined to jobs of lower power, status and pay, often within service industries roles" (Dillon, 2020). She also points out that, whereas digital assistants are typically gendered as women, digital advisors (legal, financial, medical), are typically gendered as men.…”

Rethinking ‘Sex Robots’

“…The opening statement introduces Azuma as "A companion to her hard-working master in this world, her cute personality and lovable behaviour help you relax" [sic] (2020). While 'companion' speaks to the positive emotional potential with Azuma, the presentation of femininity as servility in emergent assistive technologies raises concerns (Schiller and McMahon, 2019;Dillon, 2020). The femininity embodied by Azuma, and assigned to her as a "bride character", reflects general trends in designing digital feminities that relies on a construction of femininity as young and 'cute' (McIntyre, 2020).…”