AI Opaqueness: What Makes AI Systems More Transparent?

Abstract

Artificially Intelligent (AI) systems are pervasive, but poorly understood by their users and, at times, developers. It is often unclear how and why certain algorithms make choices, predictions, or conclusions. What does AI transparency mean? What explanations do AI system users desire? This panel discusses AI opaqueness with examples in applied context such as natural language processing, people categorization, judicial decision explanations, and system recommendations. We offer insights from interviews with AI system users about their perceptions and developers’ lessons learned. What steps should be taken towards AI transparency and accountability for its decisions? Les systèmes artificiellement intelligents (IA) sont omniprésents, mais mal compris par leurs utilisateurs et, parfois, par les développeurs. On ne sait souvent pas comment et pourquoi certains algorithmes font des choix, des prédictions ou des conclusions. Que signifie la transparence de l'IA? Quelles explications les utilisateurs du système d'IA souhaitent-ils? Ce panel examine l'opacité de l'IA avec des exemples dans un contexte appliqué tels que le traitement du langage naturel, la catégorisation des personnes, les explications des décisions judiciaires et les recommandations système. Nous proposons des informations issues d'entretiens avec des utilisateurs de systèmes d'IA sur leurs perceptions et les leçons apprises par les développeurs. Quelles mesures devraient être prises pour assurer la transparence et la responsabilité de l'IA pour ses décisions?

Untitled

“…In sustainability initiatives, it is crucial to ensure that AI-driven decisions can be explained and comprehended by stakeholders. Transparent AI systems enable individuals and communities to grasp the rationale behind decisions that impact their lives, fostering trust and acceptance (Rubin, 2020). Additionally, it is imperative to establish mechanisms for holding AI accountable, as it should not be deployed without oversight and accountability (Busuioc, 2020).…”

Philosophical paradigms in information technology research

“…Systems are said to be opaque if their internal implementation cannot be easily revealed (Rubin, 2020). Many organizations that develop AI systems do not or are not ready to reveal the algorithm that powers these systems.…”