Managing the tension between opposing effects of explainability of artificial intelligence: a contingency theory perspective

Abstract

PurposeResearch into the interpretability and explainability of data analytics and artificial intelligence (AI) systems is on the rise. However, most recent studies either solely promote the benefits of explainability or criticize it due to its counterproductive effects. This study addresses this polarized space and aims to identify opposing effects of the explainability of AI and the tensions between them and propose how to manage this tension to optimize AI system performance and trustworthiness.Design/methodology/approachThe author systematically reviews the literature and synthesizes it using a contingency theory lens to develop a framework for managing the opposing effects of AI explainability.FindingsThe author finds five opposing effects of explainability: comprehensibility, conduct, confidentiality, completeness and confidence in AI (5Cs). The author also proposes six perspectives on managing the tensions between the 5Cs: pragmatism in explanation, contextualization of the explanation, cohabitation of human agency and AI agency, metrics and standardization, regulatory and ethical principles, and other emerging solutions (i.e. AI enveloping, blockchain and AI fuzzy systems).Research limitations/implicationsAs in other systematic literature review studies, the results are limited by the content of the selected papers.Practical implicationsThe findings show how AI owners and developers can manage tensions between profitability, prediction accuracy and system performance via visibility, accountability and maintaining the “social goodness” of AI. The results guide practitioners in developing metrics and standards for AI explainability, with the context of AI operation as the focus.Originality/valueThis study addresses polarized beliefs amongst scholars and practitioners about the benefits of AI explainability versus its counterproductive effects. It poses that there is no single best way to maximize AI explainability. Instead, the co-existence of enabling and constraining effects must be managed.

Designing and Managing Human-AI Interactions

“…For that reason, questions related to accountability need to be addressed, and AI models need to be explainable and their chain of reasoning leading to an outcome need to be reproducible. One problem is that instruments of explainable AI, or interpretable machine learning techniques, are often developed by computer scientists for computer scientists (Abedin, 2021;Arrieta et al, 2020); in contrast, we argue that research on explainability needs to focus more on end-users who require different, and more approachable, forms of explanations.…”

“…As advances in AI algorithms and systems provide proxy agency to users through customization of tasks and decision, they become increasingly capable of exerting their own agency (Sundar, 2020), which in turn gives rise to the tension between machine agency and human agency (Abedin, 2021). There are different research streams on agency, many of which discuss agency only as a potential attribute of humans (e.g., Nevo et al, 2018).…”

“…Adding the mounting concern about privacy, security, and transparency of AI systems, it is therefore not surprising that scholars in IS, human-computer interaction, and related disciplines have called for more research to reconcile these tensions between AI agency and human agency (Abedin, 2021). As AI gets more autonomous and takes on more agency, AI-human interactions become ever more determined by algorithms (Sundar, 2020).…”

Designing a feature selection method based on explainable artificial intelligence

“…This may lead to more accurate ML models and, consequently, to better business outcomes. Explainability might also increase user's trust in AI systems and thereby raise technology adoption (Abedin, 2021).…”

Guest editorial: Interpretable AI-enabled online behavior analytics

“…The second article by Babak Abedin proposes a novel framework for managing the opposing effects of AI explainability and addresses polarized beliefs about the benefits of AI explainability and its counterproductive effects. It posits that there is no single best way to maximize AI explainability, and instead, the co-existence of enabling and constraining effects must be managed (Abedin, 2022).…”