Introduction to Interpretability and Explainability

Abstract

In recent years, we have seen gains in adoption of machine learning and artificial intelligence applications. However, continued adoption is being constrained by several limitations. The field of Explainable AI addresses one of the largest shortcomings of machine learning and deep learning algorithms today: the interpretability and explainability of models. As algorithms become more powerful and are better able to predict with better accuracy, it becomes increasingly important to understand how and why a prediction is made. Without interpretability and explainability, it would be difficult for us to trust the predictions of real-life applications of AI. Human-understandable explanations will encourage trust and continued adoption of machine learning systems as well as increasing system safety. As an emerging field, explainable AI will be vital for researchers and practitioners in the coming years.This book takes an in-depth approach to presenting the fundamentals of explainable AI through mathematical theory and practical use cases. The content is split into four parts: pre-model methods, intrinsic methods, post-hoc methods, and deeplearning methods. The first part introduces pre-model techniques for Explainable AI (XAI). Part Two presents classical and modern intrinsic model interpretability methods, while Part Three details the collection of post-hoc methods. Part Four dives into methods tailored specifically for deep learning models. All concepts are presented with numerous examples to build practical knowledge. This book makes an assumption that readers have some background in elementary machine learning and deep learning models. Knowledge of the python programming language and its associated packages is helpful, but not a requirement.

Identifying Cardiovascular Disease Risk Factors in Adults with Explainable Artificial Intelligence

“…When combined with Lundberg and Lee’s 30 ML algorithms, this framework, which works with Shapley values, brought transparency, explainability, and predictability to black-box models. 31 The SHAP works on all possible permutations of the features and calculates the average of the contributions. This study presents an explanatory design with SHAP algorithms for CVD risk factors.…”

Explainability and transparency in the realm of digital humanities: toward a historian XAI

“…The term XAI refers to the field of AI research that is dedicated to the generation of explanations with regard to the increasingly complex machine learning models (Montavon et al, 2018;Samek et al, 2019Samek et al, , 2021Holzinger et al, 2022) and is crucial in numerous domains to ensure model safety, robustness, and resilience to data drift. They may also reveal useful correlations in the data as well as ensure that the model results are understood by domain experts (Samek et al, 2021;Lapuschkin et al, 2019;Kamath & Liu, 2021). This field opened the door to numerous impactful contributions across an array of knowledge areas.…”

Towards Personalised Mood Prediction and Explanation for Depression from Biophysical Data

“…Without explaining why a model predicts a mood score, healthcare professionals cannot determine what insights the prediction contains [ 37 ]. These insights can then be used to check a model’s fidelity (whether the model predictions make sense) [ 38 ] and suggest interventions that help manage the symptoms in a personalised fashion.…”

“…Recent advances in explainable Artificial Intelligence (XAI) offer solutions to the problem of trustworthiness in ML and DL models. Explainable models (we use the terms explainability and interpretability interchangeably in this work [ 38 ]) such as Decision Trees [ 36 ] can be easily processed/simplified to explain their outputs [ 39 ]. However, their expressive power is limited by their size, and increasing their expressiveness decreases their interpretability.…”