Visual Concept Programming: A Visual Analytics Approach to Injecting Human Intelligence At Scale

CAN: Concept‐Aligned Neurons for Visual Comparison of Deep Neural Network Models

“…Other works leverage per‐instance concept‐based explanations [GWZK19] to analyze model behavior across a collection of images [HMKB22], while Zhao et al [ZXSR21] employ interactive visualization as a means of incrementally finding interpretable concepts. Moreover, Hoque et al [HHS*22] demonstrate the use of discovered concepts to build customized classifiers with minimal human effort. Our approach similarly takes a concept‐based approach for analyzing and comparing models, but does so at a much larger scale than prior works, in that we aim to support neuron interpretability with open vocabulary concept sets.…”

Doom or Deliciousness: Challenges and Opportunities for Visualization in the Age of Generative Models

“…This process can be time‐consuming and difficult, as data modeling is notoriously challenging [RS19]. A generative model might aid this process by helping the user identify what counts as data (or just by defining the data workspace subsequent analysis will occur in), such as by preparing a SQL query, suggesting a data model, finding a relevant data set, or otherwise injecting its own knowledge [HHS * 22]. For instance, a natural language prompt such as “get data from earnings of the fourth quarter relative to our competitors” offers ample ambiguity and opportunity for the model to exert agency, such as in identifying who the competitors are, how to compute earnings data, which business units to include, and so on.…”

ESCAPE: Countering Systematic Errors from Machine’s Blind Spots via Interactive Visual Analysis

“…These concept-level interpretability methods, however, require the human ability to observe and extract semantically meaningful concepts [20]. There are various ways to identify and extract concepts in collaboration with humans and systems [20,26,35,41,43,44,51,56,61]. ConceptExtract [61] aimed to support concept extraction and classification in a human-in-the-loop workflow and visual tools.…”

“…ConceptExplainer [56] was designed to explore the concept associations focusing on validating conceptual overlapping between classes, especially serving as a concept exploration tool for non-expert users. In [26], a self-supervised technique was proposed to automatically extract visual vocabulary to allow experts to refine the labeled data and understand the concepts.…”