Questioning the AI: Informing Design Practices for Explainable AI User Experiences

Liao, Q. Vera; Gruen, D.; Miller, Sarah

doi:10.1145/3313831.3376590

Cited by 486 publications

(255 citation statements)

References 86 publications

(134 reference statements)

Supporting

Mentioning

251

Contrasting

Unclassified

Order By: Relevance

“…When they considered the answer was complete, we used the questions in Table 1 to continue the discussion. The questions are inspired by [49], and we tailor them to our context to uncover the elements of scenarios [54].…”

Section: Using and Demonstrating The Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Scenario-Based Requirements Elicitation for User-Centric Explainable AI

Cirqueira

Nedbal

Helfert

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Explainable Artificial Intelligence (XAI) develops technical explanation methods and enable interpretability for human stakeholders on why Artificial Intelligence (AI) and machine learning (ML) models provide certain predictions. However, the trust of those stakeholders into AI models and explanations is still an issue, especially domain experts, who are knowledgeable about their domain but not AI inner workings. Social and user-centric XAI research states it is essential to understand the stakeholder's requirements to provide explanations tailored to their needs, and enhance their trust in working with AI models. Scenariobased design and requirements elicitation can help bridge the gap between social and operational aspects of a stakeholder early before the adoption of information systems and identify its real problem and practices generating user requirements. Nevertheless, it is still rarely explored the adoption of scenarios in XAI, especially in the domain of fraud detection to supporting experts who are about to work with AI models. We demonstrate the usage of scenario-based requirements elicitation for XAI in a fraud detection context, and develop scenarios derived with experts in banking fraud. We discuss how those scenarios can be adopted to identify user or expert requirements for appropriate explanations in his daily operations and to make decisions on reviewing fraudulent cases in banking. The generalizability of the scenarios for further adoption is validated through a systematic literature review in domains of XAI and visual analytics for fraud detection.

show abstract

Section: Using and Demonstrating The Methodsmentioning

confidence: 99%

“…The methodology is assessed in a hypothetical hiring scenario. In [49], a Question-driven approach to assess explanation needs is proposed. The authors adopt a taxonomy of XAI methods mapped to user question types.…”

Section: Related Workmentioning

confidence: 99%

Scenario-Based Requirements Elicitation for User-Centric Explainable AI

Cirqueira

Nedbal

Helfert

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…This controversial GDPR requirement is vague and difficult to satisfy in general, but international research efforts at developing Explainable AI (XAI) have blossomed [9,47,48,106,116,75,71]. A useful and practical resource are the three reports on "Explaining decisions made with AI" from the U.K. Information Commissioner's Office and the Alan Turing Institute [57].…”

Section: Explainable User Interfacesmentioning

confidence: 99%

Bridging the Gap Between Ethics and Practice

Shneiderman

2020

ACM Trans. Interact. Intell. Syst.

366

View full text Add to dashboard Cite

This article attempts to bridge the gap between widely discussed ethical principles of Human-centered AI (HCAI) and practical steps for effective governance. Since HCAI systems are developed and implemented in multiple organizational structures, I propose 15 recommendations at three levels of governance: team, organization, and industry. The recommendations are intended to increase the reliability, safety, and trustworthiness of HCAI systems: (1) reliable systems based on sound software engineering practices, (2) safety culture through business management strategies, and (3) trustworthy certification by independent oversight. Software engineering practices within teams include audit trails to enable analysis of failures, software engineering workflows, verification and validation testing, bias testing to enhance fairness, and explainable user interfaces. The safety culture within organizations comes from management strategies that include leadership commitment to safety, hiring and training oriented to safety, extensive reporting of failures and near misses, internal review boards for problems and future plans, and alignment with industry standard practices. The trustworthiness certification comes from industry-wide efforts that include government interventions and regulation, accounting firms conducting external audits, insurance companies compensating for failures, nongovernmental and civil society organizations advancing design principles, and professional organizations and research institutes developing standards, policies, and novel ideas. The larger goal of effective governance is to limit the dangers and increase the benefits of HCAI to individuals, organizations, and society. CCS Concepts: • Computing methodologies → Philosophical/theoretical foundations of artificial intelligence; • Human-centered computing → HCI theory, concepts and models; • Software and its engineering → Software development process management;

show abstract

“…The practical approach aims to limit the number of syntheses and assays needed to find and optimize new hit and lead compounds, especially when elaborate and expensive tests are performed. XAI-assisted drug design is expected to help overcome some of these issues, by allowing to take informed action while simultaneously considering medicinal chemistry knowledge, model logic and awareness on the system's limitations 39 . XAI will foster the collaboration between medicinal chemists, chemoinformaticians and data scientists 40,41 .…”

Section: Drug Discovery With Explainable Artificial Intelligencementioning

confidence: 99%

Drug discovery with explainable artificial intelligence

2020

View full text Add to dashboard Cite

arious concepts of 'artificial intelligence' (AI) have been successfully adopted for computer-assisted drug discovery in the past few years 1-3. This advance is mostly owed to the ability of deep learning algorithms, that is, artificial neural networks with multiple processing layers, to model complex nonlinear inputoutput relationships, and perform pattern recognition and feature extraction from low-level data representations. Certain deep learning models have been shown to match or even exceed the performance of the familiar existing machine learning and quantitative structure-activity relationship (QSAR) methods for drug discovery 4-6. Moreover, deep learning has boosted the potential and broadened the applicability of computer-assisted discovery, for example, in molecular design 7,8 , chemical synthesis planning 9,10 , protein structure prediction 11 and macromolecular target identification 12,13. The ability to capture intricate nonlinear relationships between input data (for example, chemical structure representations) and the associated output (for example, assay readout) often comes at the price of limited comprehensibility of the resulting model. While there have been efforts to explain QSARs in terms of algorithmic insights and molecular descriptor analysis 14-19 , deep neural network models notoriously elude immediate accessibility by the human mind 20. In medicinal chemistry in particular, the availability of 'rules of thumb' correlating biological effects with physicochemical properties underscores the willingness, in certain situations, to sacrifice accuracy in favour of models that better fit human intuition 21-23. Thus, blurring the lines between the 'two QSARs' 24 (that is, mechanistically interpretable versus highly accurate models) may be key to accelerated drug discovery with AI 25. Automated analysis of medical and chemical knowledge to extract and represent features in a human-intelligible format dates back to the 1990s 26,27 , but has been receiving increasing attention due to the re-emergence of neural networks in chemistry and healthcare. Given the current pace of AI in drug discovery and related fields, there will be an increased demand for methods that help us understand and interpret the underlying models. In an effort to mitigate the lack of interpretability of certain machine learning models, and to augment human reasoning and decision-making, 28 , attention has been drawn to explainable AI (XAI) approaches 29,30. Providing informative explanations alongside the mathematical models aims to (1) render the underlying decision-making process transparent ('understandable') 31 , (2) avoid correct predictions for the wrong reasons (the so-called clever Hans effect) 32 , (3) avert unfair biases or unethical discrimination 33 and (4) bridge the gap between the machine learning community and other scientific disciplines. In addition, effective XAI can help scientists navigate 'cognitive valleys' 28 , allowing them to hone their knowledge and beliefs on the investigated process 34. While the e...

show abstract

Questioning the AI: Informing Design Practices for Explainable AI User Experiences

Cited by 486 publications

References 86 publications

Scenario-Based Requirements Elicitation for User-Centric Explainable AI

Scenario-Based Requirements Elicitation for User-Centric Explainable AI

Bridging the Gap Between Ethics and Practice

Drug discovery with explainable artificial intelligence

Contact Info

Product

Resources

About