RuleMatrix: Visualizing and Understanding Classifiers with Rules

127

t-Distributed Stochastic Neighbor Embedding (t-SNE) for the visualization of multidimensional data has proven to be a popular approach, with successful applications in a wide range of domains. Despite their usefulness, t-SNE projections can be hard to interpret or even misleading, which hurts the trustworthiness of the results. Understanding the details of t-SNE itself and the reasons behind specific patterns in its output may be a daunting task, especially for non-experts in dimensionality reduction. In this work, we present t-viSNE, an interactive tool for the visual exploration of t-SNE projections that enables analysts to inspect different aspects of their accuracy and meaning, such as the effects of hyper-parameters, distance and neighborhood preservation, densities and costs of specific neighborhoods, and the correlations between dimensions and visual patterns. We propose a coherent, accessible, and well-integrated collection of different views for the visualization of t-SNE projections. The applicability and usability of t-viSNE are demonstrated through hypothetical usage scenarios with real data sets. Finally, we present the results of a user study where the tool's effectiveness was evaluated. By bringing to light information that would normally be lost after running t-SNE, we hope to support analysts in using t-SNE and making its results better understandable.

Section: Use Casesmentioning

confidence: 99%

t-viSNE: Interactive Assessment and Interpretation of t-SNE Projections

Chatzimparmpas

Martins

Kerren

2020

127

“…Many other systems are highly specialized, focusing on just one aspect of model understanding. En-sembleMatrix [31] is designed for comparing models that make up ensembles, while RuleMatrix [21] attempts to explain ML models in terms of simple rules.…”

Section: Model Understanding Frameworkmentioning

confidence: 99%

“…Understanding a model can depend on a user's ability to generate explanations for model behavior at an instance-, feature-and subgrouplevel. Despite attempts to use visualizations for generating these explanations via rules [21], analogies, and layer activations [20], most approaches run into issues of visual complexity, the need to support multiple exploratory workflows, and reliance on simple, interpretable data for meaningful insights [17,20,21,34]. WIT should be able to provide multiple, complementary visualizations through which users can arrive at and validate generalized explanations for model behavior locally and globally.…”

Section: User Needsmentioning

confidence: 99%

The What-If Tool: Interactive Probing of Machine Learning Models

Wexler

Pushkarna

Bolukbasi

et al. 2019

323

261

A key challenge in developing and deploying Machine Learning (ML) systems is understanding their performance across a wide range of inputs. To address this challenge, we created the What-If Tool, an open-source application that allows practitioners to probe, visualize, and analyze ML systems, with minimal coding. The What-If Tool lets practitioners test performance in hypothetical situations, analyze the importance of different data features, and visualize model behavior across multiple models and subsets of input data. It also lets practitioners measure systems according to multiple ML fairness metrics. We describe the design of the tool, and report on real-life usage at different organizations. Index Terms-Interactive Machine Learning, Model Debugging, Model Comparison

“…Recent research in visual analytics integrated machine learning tools with intuitive and manipulable representation and interface. Examples include RuleMatrix [29]'s rule-based visual interface for explaining decision rules based on the secondary decision tree model, the distributionbased visual representation for global-level explanation, and Rivelo's instance-and feature-level representation [38]. Mainfold [42] suggested a model-agnostic framework to interpret the outcome, inspect a subset of instances, and refine the feature set or model, to facilitate the comparison of models.…”

Section: Explainable Machine Learningmentioning

confidence: 99%

FairSight: Visual Analytics for Fairness in Decision Making

Ahn

Lin

2019

Fig. 1. We propose a design framework to protect individuals and groups from discrimination in algorithm-assisted decision making. A visual analytic system, FairSight, is implemented based on our proposed framework, to help data scientists and practitioners make fair decisions. The decision is made through ranking individuals who are either members of a protected group (orange bars) or a non-protected group (green bars). (a) The system provides a pipeline to help users understand the possible bias in a machine learning task as a mapping from the input space to the output space. (b) Different notions of fairnessindividual fairness and group fairnessare measured and summarized numerically and visually. For example, the individual fairness is quantified by how pairwise distances between individuals are preserved through the mapping. The group fairness is quantified by the extent to which it leads to fair outcome distribution across groups, with (i) a 2D plot, (ii) a color-coded matrix , and (iii) a ranked-list plot capturing the pattern of potential biases.The system provides diagnostic modules to help (iv) identify and (v) mitigate biases through (c) investigating features before running a model, and (d) leveraging fairness-aware algorithms during and after the training step.Abstract-Data-driven decision making related to individuals has become increasingly pervasive, but the issue concerning the potential discrimination has been raised by recent studies. In response, researchers have made efforts to propose and implement fairness measures and algorithms, but those efforts have not been translated to the real-world practice of data-driven decision making. As such, there is still an urgent need to create a viable tool to facilitate fair decision making. We propose FairSight, a visual analytic system to address this need; it is designed to achieve different notions of fairness in ranking decisions through identifying the required actions -understanding, measuring, diagnosing and mitigating biases -that together lead to fairer decision making. Through a case study and user study, we demonstrate that the proposed visual analytic and diagnostic modules in the system are effective in understanding the fairness-aware decision pipeline and obtaining more fair outcomes.