Bias and Epistemic Injustice in Conversational AI

Laacke, Sebastian

doi:10.1080/15265161.2023.2191055

Cited by 11 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ethical concerns with CAIs in therapeutic contexts include the biases and other harmful prompts that might arise in such exchanges, along with the potential dearth of responsibility and accountability for these harms [6][7][8][9][10]. However, even if such patent ethical concerns were addressed or eradicated, central questions would persist: what sort of presence or entity do we have in CAIs?…”

Section: "Speaking To Machines": Cais and The Possibility Of Insightmentioning

confidence: 99%

The Machine Speaks: Conversational AI and the Importance of Effort to Relationships of Meaning

Hartford,

Stein

2024

JMIR Ment Health

View full text Add to dashboard Cite

The focus of debates about conversational artificial intelligence (CAI) has largely been on social and ethical concerns that arise when we speak to machines—what is gained and what is lost when we replace our human interlocutors, including our human therapists, with AI. In this viewpoint, we focus instead on a distinct and growing phenomenon: letting machines speak for us. What is at stake when we replace our own efforts at interpersonal engagement with CAI? The purpose of these technologies is, in part, to remove effort, but effort has enormous value, and in some cases, even intrinsic value. This is true in many realms, but especially in interpersonal relationships. To make an effort for someone, irrespective of what that effort amounts to, often conveys value and meaning in itself. We elaborate on the meaning, worth, and significance that may be lost when we relinquish effort in our interpersonal engagements as well as on the opportunities for self-understanding and growth that we may forsake.

show abstract

Section: "Speaking To Machines": Cais and The Possibility Of Insightmentioning

confidence: 99%

The Machine Speaks: Conversational AI and the Importance of Effort to Relationships of Meaning

Hartford,

Stein

2024

JMIR Ment Health

View full text Add to dashboard Cite

show abstract

“…An LLM-enhanced CAI may not encode cultural nuances and the uniqueness of individual experiences in its outputs while its biases significantly influence how the system presents and discusses knowledge with patients. This can contribute to “epistemic injustice” [ 60 ], making individuals with depression potentially feel more isolated and their perspectives undervalued and misunderstood. In addition, human experts—for instance, psychiatrists—have epistemic duties, including being truthful and justifying their beliefs [ 43 ].…”

Section: Use Of Llm-enhanced Cais By Individuals With Depression: 2 C...mentioning

confidence: 99%

The Role of Humanization and Robustness of Large Language Models in Conversational Artificial Intelligence for Individuals With Depression: A Critical Analysis

Ferrario,

Sedlakova,

Trachsel

2024

JMIR Ment Health

View full text Add to dashboard Cite

Large language model (LLM)–powered services are gaining popularity in various applications due to their exceptional performance in many tasks, such as sentiment analysis and answering questions. Recently, research has been exploring their potential use in digital health contexts, particularly in the mental health domain. However, implementing LLM-enhanced conversational artificial intelligence (CAI) presents significant ethical, technical, and clinical challenges. In this viewpoint paper, we discuss 2 challenges that affect the use of LLM-enhanced CAI for individuals with mental health issues, focusing on the use case of patients with depression: the tendency to humanize LLM-enhanced CAI and their lack of contextualized robustness. Our approach is interdisciplinary, relying on considerations from philosophy, psychology, and computer science. We argue that the humanization of LLM-enhanced CAI hinges on the reflection of what it means to simulate “human-like” features with LLMs and what role these systems should play in interactions with humans. Further, ensuring the contextualization of the robustness of LLMs requires considering the specificities of language production in individuals with depression, as well as its evolution over time. Finally, we provide a series of recommendations to foster the responsible design and deployment of LLM-enhanced CAI for the therapeutic support of individuals with depression.

show abstract

“…Conversational AI may therefore simply disseminate what works for largely non-Hispanic White populations, perpetuating existing disparities. There also may be biases embedded in the conversational scripts generated by CAI platforms that reflect the communication practices of the dominant social group . The proprietary algorithms of CAI platforms should not be opaque but accessible and interrogated.…”

Section: Equity and Transparencymentioning

confidence: 99%

“…There also may be biases embedded in the conversational scripts generated by CAI platforms that reflect the communication practices of the dominant social group. 9 The proprietary algorithms of CAI platforms should not be opaque but accessible and interrogated.…”

Section: Equity and Transparencymentioning

confidence: 99%