State of the Art of Visual Analytics for eXplainable Deep Learning

Rosa, Biagio La; Blasilli, Graziano; Bourqui, Romain; Santucci, Giuseppe; Capobianco, Roberto; Bertini, Enrico; Giot, Romain; Angelini, Marco

doi:10.1111/cgf.14733

Cited by 33 publications

(8 citation statements)

References 207 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The case study is user-agnostic and directed towards any ML user category [50], [51], such as practitioners (e.g., Architects, Trainers) or end users. Users can benefit from the insights gained in Phase 10 in a variety of ways, depending on their role and goals: both practitioners (e.g., machine learning experts, data scientists) and end users (e.g., clinicians, domain experts) will benefit from understanding how features affect predictions.…”

Section: Phasementioning

confidence: 99%

A Visual Analytics Conceptual Framework for Explorable and Steerable Partial Dependence Analysis

Angelini

Blasilli

Lenti

et al. 2024

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Machine learning techniques are a driving force for research in various fields, from credit card fraud detection to stock analysis. Recently, a growing interest in increasing human involvement has emerged, with the primary goal of improving the interpretability of machine learning models. Among different techniques, Partial Dependence Plots (PDP) represent one of the main model-agnostic approaches for interpreting how the features influence the prediction of a machine learning model. However, its limitations (i.e., visual interpretation, aggregation of heterogeneous effects, inaccuracy, and computability) could complicate or misdirect the analysis. Moreover, the resulting combinatorial space can be challenging to explore both computationally and cognitively when analyzing the effects of more features at the same time. This paper proposes a conceptual framework that enables effective analysis workflows, mitigating state-of-the-art limitations. The proposed framework allows for exploring and refining computed partial dependences, observing incrementally accurate results, and steering the computation of new partial dependences on user-selected subspaces of the combinatorial and intractable space. With this approach, the user can save both computational and cognitive costs, in contrast with the standard monolithic approach that computes all the possible combinations of features on all their domains in batch. The framework is the result of a careful design process involving experts' knowledge during its validation and informed the development of a prototype, W4SP (available at https://aware-diag-sapienza.github.io/W4SP/), that demonstrates its applicability traversing its different paths. A case study shows the advantages of the proposed approach.

show abstract

Section: Phasementioning

confidence: 99%

A Visual Analytics Conceptual Framework for Explorable and Steerable Partial Dependence Analysis

Angelini

Blasilli

Lenti

et al. 2024

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…An extensive body of existing work leverages VA systems and techniques to understand, assess, and debug deep‐learning models in various domains [HKPC19,LRBB*23, GZL*21,SSSEA20]. Earlier works in NLP include sample‐level (local) explanation techniques e.g., saliency visualization of encodings [LCHJ16], bipartite‐graph attention visualization in LLMs [Vig19], and feature importance saliency visualizations based on LIME [RSG16] and SHAP [LL17].…”

Section: Related Workmentioning

confidence: 99%

Visual Analytics for Fine‐grained Text Classification Models and Datasets

Battogtokh,

Xing,

Davidescu

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

In natural language processing (NLP), text classification tasks are increasingly fine‐grained, as datasets are fragmented into a larger number of classes that are more difficult to differentiate from one another. As a consequence, the semantic structures of datasets have become more complex, and model decisions more difficult to explain. Existing tools, suited for coarse‐grained classification, falter under these additional challenges. In response to this gap, we worked closely with NLP domain experts in an iterative design‐and‐evaluation process to characterize and tackle the growing requirements in their workflow of developing fine‐grained text classification models. The result of this collaboration is the development of SemLa, a novel Visual Analytics system tailored for 1) dissecting complex semantic structures in a dataset when it is spatialized in model embedding space, and 2) visualizing fine‐grained nuances in the meaning of text samples to faithfully explain model reasoning. This paper details the iterative design study and the resulting innovations featured in SemLa. The final design allows contrastive analysis at different levels by unearthing lexical and conceptual patterns including biases and artifacts in data. Expert feedback on our final design and case studies confirm that SemLa is a useful tool for supporting model validation and debugging as well as data annotation.

show abstract

“…Developing algorithms and techniques to explain the behavior of classifiers is an active area of research [AS22, RBB*23, AASA21, SMV*19]. This applies especially to so‐called Deep Learning techniques which are notorious for their “black‐box” aspect: the models thus learned consist of sets of opaque numbers arranged in vectors and matrices and combined through special operations.…”

Section: Background and Related Workmentioning

confidence: 99%

Exploring Classifiers with Differentiable Decision Boundary Maps

Machado,

Behrisch,

Telea

2024

Computer Graphics Forum

View full text Add to dashboard Cite

Explaining Machine Learning (ML) — and especially Deep Learning (DL) — classifiers' decisions is a subject of interest across fields due to the increasing ubiquity of such models in computing systems. As models get increasingly complex, relying on sophisticated machinery to recognize data patterns, explaining their behavior becomes more difficult. Directly visualizing classifier behavior is in general infeasible, as they create partitions of the data space, which is typically high dimensional. In recent years, Decision Boundary Maps (DBMs) have been developed, taking advantage of projection and inverse projection techniques. By being able to map 2D points back to the data space and subsequently run a classifier, DBMs represent a slice of classifier outputs. However, we recognize that DBMs without additional explanatory views are limited in their applicability. In this work, we propose augmenting the naive DBM generating process with views that provide more in‐depth information about classifier behavior, such as whether the training procedure is locally stable. We describe our proposed views — which we term Differentiable Decision Boundary Maps — over a running example, explaining how our work enables drawing new and useful conclusions from these dense maps. We further demonstrate the value of these conclusions by showing how useful they would be in carrying out or preventing a dataset poisoning attack. We thus provide evidence of the ability of our proposed views to make DBMs significantly more trustworthy and interpretable, increasing their utility as a model understanding tool.

show abstract

State of the Art of Visual Analytics for eXplainable Deep Learning

Cited by 33 publications

References 207 publications

A Visual Analytics Conceptual Framework for Explorable and Steerable Partial Dependence Analysis

A Visual Analytics Conceptual Framework for Explorable and Steerable Partial Dependence Analysis

Visual Analytics for Fine‐grained Text Classification Models and Datasets

Exploring Classifiers with Differentiable Decision Boundary Maps

Contact Info

Product

Resources

About