ConceptExplainer: Understanding the Mental Model of Deep Learning Algorithms via Interactive Concept-based Explanations

Huang, Jinbin; Mishra, Aditi; Kwon, Bum Chul; Bryan, Chris

doi:10.48550/arxiv.2204.01888

Cited by 4 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, users could define custom difference metrics [51] to study desired behavior. Although ProactiV supports analysis of local model behavior with the instance optimization summary and input parameter distributions, this analysis can be augmented with advanced concept-based or semantic analysis [25], [26]. Such an analysis could support a more detailed analysis of image semantics.…”

Section: Discussion and Future Workmentioning

confidence: 99%

“…Concept-based analysis has been proposed to support the identification of failure patterns. Methods tag images with human-interpretable concept(s), like "shadow", either interactively [25] or via automatic concept-based explanations [26]. ConceptExplainer [26] further supported navigating the complex concept space for finer model behavior analysis, revealing semantic causes of failure.…”

Section: Related Workmentioning

confidence: 99%

“…Methods tag images with human-interpretable concept(s), like "shadow", either interactively [25] or via automatic concept-based explanations [26]. ConceptExplainer [26] further supported navigating the complex concept space for finer model behavior analysis, revealing semantic causes of failure. For example, a model may fail to detect objects under a "shadow" [25].…”

Section: Related Workmentioning

confidence: 99%

“…Hence, studying global effects under complex co-occurring input transformations and in cases where classes do not exist, such as image-image translation, is challenging. Moreover, the current work focuses on classification [25], [26], [28], [29], [37], [38], object detection [25], [39] and segmentation tasks [40]. However, tasks such as imageto-image translation [8], where each output is high-dimensional and continuous, are hardly supported.…”

Section: Related Workmentioning

confidence: 99%

“…Visual analytics (VA) has shown promise in addressing these needs. VA systems support outlier detection [23], [24] and concept-based analysis [25], [26], [27]. However, data is often sparsely distributed, and it is challenging to identify the exact model breakpoints under real-world input transformations.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

ProactiV: Studying deep learning model behavior under input transformations

Prasad¹,

Sloun²,

Vilanova³

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>Deep learning (DL) models have shown performance benefits across many applications, from classification to, more recently, image-to-image translation. However, low interpretability often leads to unexpected model behavior when deployed in the real world. Usually, this unexpected behavior is because the training data domain does not reflect the deployment data domain. Identifying model breaking points under input conditions and domain shifts, i.e., input transformations, is essential to improve models. Although visual analytics (VA) has shown promise in studying the behavior of model outputs under continually varying inputs, existing methods mainly focus on per class or instance-level analysis. We aim to generalize beyond classification where classes do not exist and provide a global holistic model behavior overview under co-occurring input transformations. We present a DL model-agnostic VA method (ProactiV ) to help model developers proactively study output behavior under input transformations to identify and verify breakpoints. ProactiV relies on a proposed input optimization method to determine how far a given transformed input has to move in the input data manifold to achieve the desired output. The data from this optimization process allows the study of global and local model output behavior under input transformations at scale. Additionally, the optimization method provides insight into the input characteristics that result in desired outputs and helps recognize model biases. We highlight how ProactiV effectively supports studying model behavior under input transformations to identify and verify model breakpoints with example classification and image-to-image translation tasks.</p>

show abstract

Section: Discussion and Future Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

ProactiV: Studying deep learning model behavior under input transformations

Prasad¹,

Sloun²,

Vilanova³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

ProactiV: Studying Deep Learning Model Behavior Under Input Transformations

Prasad

Sloun

Vilanova

et al. 2024

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Deep learning (DL) models have shown performance benefits across many applications, from classification to image-to-image translation. However, low interpretability often leads to unexpected model behavior once deployed in the real world. Usually, this unexpected behavior is because the training data domain does not reflect the deployment data domain. Identifying a model's breaking points under input conditions and domain shifts, i.e., input transformations, is essential to improve models. Although visual analytics (VA) has shown promise in studying the behavior of model outputs under continually varying inputs, existing methods mainly focus on per-class or instance-level analysis. We aim to generalize beyond classification where classes do not exist and provide a global view of model behavior under co-occurring input transformations. We present a DL model-agnostic VA method (ProactiV) to help model developers proactively study output behavior under input transformations to identify and verify breaking points. ProactiV relies on a proposed input optimization method to determine the changes to a given transformed input to achieve the desired output. The data from this optimization process allows the study of global and local model behavior under input transformations at scale. Additionally, the optimization method provides insights into the input characteristics that result in desired outputs and helps recognize model biases. We highlight how ProactiV effectively supports studying model behavior with example classification and image-to-image translation tasks.

show abstract

ESCAPE: Countering Systematic Errors from Machine’s Blind Spots via Interactive Visual Analysis

Ahn

Lin

et al. 2023

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Trovato and Tobin, et al. approaches, relative concept association to better quantify the associations between a concept and instances, and debias method to mitigate spurious associations. We demonstrate the utility of our proposed ESCAPE system and statistical measures through extensive evaluation including quantitative experiments, usage scenarios, expert interviews, and controlled user experiments. CCS CONCEPTS• Human-centered computing → Visualization toolkits; Visualization systems and tools.

show abstract

ConceptExplainer: Understanding the Mental Model of Deep Learning Algorithms via Interactive Concept-based Explanations

Cited by 4 publications

References 0 publications

ProactiV: Studying deep learning model behavior under input transformations

ProactiV: Studying deep learning model behavior under input transformations

ProactiV: Studying Deep Learning Model Behavior Under Input Transformations

ESCAPE: Countering Systematic Errors from Machine’s Blind Spots via Interactive Visual Analysis

Contact Info

Product

Resources

About