TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations

Abstract

Practitioners increasingly use machine learning (ML) models, yet they have become more complex and harder to understand. To address this issue, researchers have proposed techniques to explain model predictions. However, practitioners struggle to use explainability methods because they do not know which to choose and how to interpret the results. We address these challenges by introducing TalkToModel: an interactive dialogue system that enables users to explain ML models through natural language conversations. TalkToModel comprises three components: 1) an adaptive dialogue engine that interprets natural language and generates meaningful responses, 2) an execution component, which constructs the explanations used in the conversation, 3) a conversational interface. In real-world evaluations, 73% of healthcare workers agreed they would use TalkToModel over existing systems for understanding a disease prediction model, and 85% of ML professionals agreed TalkToModel was easier to use, demonstrating that TalkToModel is highly effective for model explainability.

Building trustworthy NeuroSymbolic AI Systems: Consistency, reliability, explainability, and safety

“…These techniques establish connections between the LM's attention patterns and concepts sourced from understandable knowledge repositories. Within this approach, two methods have emerged: (a) Attribution scores and LM tuning (Slack et al 2023) and Factual Knowledge-based Scoring and LM tuning (Yang, Chen, et al 2023;Sun et al 2023). The latter method holds particular significance in health and wellbeing because it provides explainability for clinicians as users.…”

“…Within this approach, two methods have emerged: (a) Attribution scores and LM tuning (Slack et al. 2023) and Factual Knowledge‐based Scoring and LM tuning (Yang, Chen, et al. 2023; Sun et al.…”

A Comparative Analysis of Large Language Models to Evaluate Robustness and Reliability in Adversarial Conditions

“…Current evaluation metrics might not fully capture the subtleties of model robustness, calling for the development of more comprehensive measures [34]- [36]. The impact of model interpretability on robustness has been another focus, with evidence suggesting that more interpretable models can be easier to defend against adversarial attacks [16], [37], [38]. Lastly, the exploration of model robustness across different languages and domains revealed that robustness levels are not uniform, indicating a need for domain-specific strategies to enhance resilience [39].…”

Measuring the Interpretability and Explainability of Model Decisions of Five Large Language Models

“…The development and implementation of frameworks for model explainability have been a focal point of research efforts. Such frameworks aim to provide users with understandable explanations for the decisions made by LLMs, often through the generation of natural language explanations or visualizations of the decision path [38,36,28,41,63,64,65]. It was observed that those frameworks significantly improve user trust and satisfaction, particularly in domains where understanding AI decisions is critical [65,27,66,56,57].…”