What do You Mean? Interpreting Image Classification with Crowdsourced Concept Extraction and Analysis

Abstract

Global interpretability is a vital requirement for image classification applications. Existing interpretability methods mainly explain a model behavior by identifying salient image patches, which require manual efforts from users to make sense of, and also do not typically support model validation with questions that investigate multiple visual concepts. In this paper, we introduce a scalable human-inthe-loop approach for global interpretability. Salient image areas identified by local interpretability methods are annotated with semantic concepts, which are then aggregated into a tabular representation of images to facilitate automatic statistical analysis of model behavior. We show that this approach answers interpretability needs for both model validation and exploration, and provides semantically more diverse, informative, and relevant explanations while still allowing for scalable and cost-efficient execution. CCS CONCEPTS• Information systems → Crowdsourcing; • Human-centered computing → Collaborative and social computing.

How can Explainability Methods be Used to Support Bug Identification in Computer Vision Models?

“…TCAV [27], ACE [14]), or textual information (e.g. SECA [8]). We use these categorizations to identify the explanations relevant to include in our probe.…”

“…Yet, limited efort has been devoted to investigating the debugging practices of computer vision practitioners. The machine learning community develops various explainability methods, often arguing their usefulness for model bugs identifcation [8,14,27,40,47]. However, few studies investigate their concrete uses in this process.…”

What Should You Know? A Human-In-the-Loop Approach to Unknown Unknowns Characterization in Image Recognition

“…The most closely related work, as we discussed, is Lakkaraju et al [27] and Liu et al [29] that use HItL methods for unknown unknowns detection. Recent work that has directly inspired ours is Balayn et al [6] that propose to use human computation to interpret the behavior of image classifiers by attaching semantic concepts to the saliency maps of classification. We employ this method for unknown unknowns characterization in image recognition, and take a step further to show that by including human specified requirements of what a model should know, we can significantly improve unknown unknowns characterization.…”

“…For effective characterization of unknown unknowns, two types of knowledge are needed: knowledge of what a model has learned, that we henceforth refer to as REALLY-KNOWS, and what a model should have learned, referred to as SHOULD-KNOW. Recent work on human-in-the-loop machine learning interpretability [6] has shown the important role of humans as computational agents to describe REALLY-KNOWS, by annotating salient image areas in image recognition with semantic concepts. In this paper, we advocate another view to the role of humans as contributors who can shed light on SHOULD-KNOW.…”

Automatic Identification of Harmful, Aggressive, Abusive, and Offensive Language on the Web: A Survey of Technical Biases Informed by Psychology Literature

“…Most issues are ultimately questions of ill-defined requirements. Developing methods to better identify the requirements of the systems prior to their development, and to test for such requirements, would allow to foresee such issues and possibly correct for them [23]. A recent study (not from the OCL domain) refers to adjacent problems as underspecification of machine learning models [76], i.e., models trained on the same dataset with the same architecture but various seemingly "unimportant" hyperparameters (e.g., initialization seed) provide similar performance on a test set, but diverging performance on the deployment data.…”