Robust Semantic Interpretability: Revisiting Concept Activation Vectors

Pfau, Jacob; Young, A. E.; Wei, Jerome; Wei, Maria L.; Keiser, Michael J.

doi:10.48550/arxiv.2104.02768

Cited by 3 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors of (Kim et al 2018) define a Concept Activation Vector as the normal to a hyperplane separating samples without a concept and samples with a concept in the model's latent activations for a selected layer. This hyperplane is commonly computed by solving a classification problem, e.g., using SVMs, ridge, lasso, or logistic regression (Kim et al 2018;Pfau et al 2021;Anders et al 2022;Yuksekgonul, Wang, and Zou 2023). We refer to Appendix A.4 for details on optimizer differences.…”

Section: Choosing the Right Directionmentioning

confidence: 99%

From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space

Dreyer,

Pahde,

Anders

et al. 2024

AAAI

View full text Add to dashboard Cite

Deep Neural Networks are prone to learning spurious correlations embedded in the training data, leading to potentially biased predictions. This poses risks when deploying these models for high-stake decision-making, such as in medical applications. Current methods for post-hoc model correction either require input-level annotations which are only possible for spatially localized biases, or augment the latent feature space, thereby hoping to enforce the right reasons. We present a novel method for model correction on the concept level that explicitly reduces model sensitivity towards biases via gradient penalization. When modeling biases via Concept Activation Vectors, we highlight the importance of choosing robust directions, as traditional regression-based approaches such as Support Vector Machines tend to result in diverging directions. We effectively mitigate biases in controlled and real-world settings on the ISIC, Bone Age, ImageNet and CelebA datasets using VGG, ResNet and EfficientNet architectures. Code and Appendix are available on https://github.com/frederikpahde/rrclarc.

show abstract

Section: Choosing the Right Directionmentioning

confidence: 99%

From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space

Dreyer,

Pahde,

Anders

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…In general, there is no guarantee that the symbolic and sub-symbolic representations Ĥ and M capture exactly the same information. This introduces a faithfulness issue, meaning that CBE explanations may not portray a reliable picture of the model's inference process [11,[48][49][50].…”

Section: Machine Representations: the Post Hoc Casementioning

confidence: 99%

Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning

Marconato,

Passerini,

Teso

2023

Entropy

View full text Add to dashboard Cite

Research on Explainable Artificial Intelligence has recently started exploring the idea of producing explanations that, rather than being expressed in terms of low-level features, are encoded in terms of interpretable concepts learned from data. How to reliably acquire such concepts is, however, still fundamentally unclear. An agreed-upon notion of concept interpretability is missing, with the result that concepts used by both post hoc explainers and concept-based neural networks are acquired through a variety of mutually incompatible strategies. Critically, most of these neglect the human side of the problem: a representation is understandable only insofar as it can be understood by the human at the receiving end. The key challenge in human-interpretable representation learning (hrl) is how to model and operationalize this human element. In this work, we propose a mathematical framework for acquiring interpretable representations suitable for both post hoc explainers and concept-based neural networks. Our formalization of hrl builds on recent advances in causal representation learning and explicitly models a human stakeholder as an external observer. This allows us derive a principled notion of alignment between the machine’s representation and the vocabulary of concepts understood by the human. In doing so, we link alignment and interpretability through a simple and intuitive name transfer game, and clarify the relationship between alignment and a well-known property of representations, namely disentanglement. We also show that alignment is linked to the issue of undesirable correlations among concepts, also known as concept leakage, and to content-style separation, all through a general information-theoretic reformulation of these properties. Our conceptualization aims to bridge the gap between the human and algorithmic sides of interpretability and establish a stepping stone for new research on human-interpretable representations.

show abstract

“…Kim et al [13] introduce Concept Activation Vectors (CAVs) that use directional derivatives to represent human-understandable concepts from a model's activations and quantify the influence of a concept on the predictions of a single target class. Pfau et al [14] build on TCAV by providing global and local conceptual sensitivities and accounting for the non-linear influence of concepts on a model's predictions. Lucieri et al [15] explore TCAV in the context of skin lesions classification using an InceptionV4 model built by the REasoning for COmplex Data (RECOD) Lab.…”

Section: A Interpretable Aimentioning

confidence: 99%

Interpretability for Multimodal Emotion Recognition using Concept Activation Vectors

Asokan¹,

Kumar²,

Ragam³

et al. 2022

Preprint

View full text Add to dashboard Cite

Multimodal Emotion Recognition refers to the classification of input video sequences into emotion labels based on multiple input modalities (usually video, audio and text). In recent years, Deep Neural networks have shown remarkable performance in recognizing human emotions, and are on par with human-level performance on this task. Despite the recent advancements in this field, emotion recognition systems are yet to be accepted for real world setups due to the obscure nature of their reasoning and decision-making process. Most of the research in this field deals with novel architectures to improve the performance for this task, with a few attempts at providing explanations for these models' decisions. In this paper, we address the issue of interpretability for neural networks in the context of emotion recognition using Concept Activation Vectors (CAVs). To analyse the model's latent space, we define humanunderstandable concepts specific to Emotion AI and map them to the widely-used IEMOCAP multimodal database. We then evaluate the influence of our proposed concepts at multiple layers of the Bi-directional Contextual LSTM (BC-LSTM) network to show that the reasoning process of neural networks for emotion recognition can be represented using human-understandable concepts. Finally, we perform hypothesis testing on our proposed concepts to show that they are significant for interpretability of this task.

show abstract

Robust Semantic Interpretability: Revisiting Concept Activation Vectors

Cited by 3 publications

References 20 publications

From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space

From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space

Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning

Interpretability for Multimodal Emotion Recognition using Concept Activation Vectors

Contact Info

Product

Resources

About