Improving Fairness in Machine Learning Systems

Holstein, Kenneth; Vaughan, J.; Daumé, Hal; Dudík, Miro; Wallach, Hanna

doi:10.1145/3290605.3300830

Cited by 571 publications

(496 citation statements)

References 57 publications

Supporting

Mentioning

449

Contrasting

Unclassified

Order By: Relevance

“…Finally, it is crucial to develop ways to systematically evaluate collaborative interfaces and to investigate the implications of designing algorithmic interactions with humans [82,83]. While an interpretabilityfirst approach could assist in highlighting fairness and bias issues in data or models [34,38], it could also introduce unwanted biases by guiding the user towards what the model has learned [4]. It is thus insufficient to limit the evaluation of a system to measures of efficiency and accuracy.…”

Section: Discussionmentioning

confidence: 99%

Visual Interaction with Deep Learning Models through Collaborative Semantic Inference

Gehrmann

Strobelt

Kruger³

et al. 2019

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Automation of tasks can have critical consequences when humans lose agency over decision processes. Deep learning models are particularly susceptible since current black-box approaches lack explainable reasoning. We argue that both the visual interface and model structure of deep learning systems need to take into account interaction design. We propose a framework of collaborative semantic inference (CSI) for the co-design of interactions and models to enable visual collaboration between humans and algorithms. The approach exposes the intermediate reasoning process of models which allows semantic interactions with the visual metaphors of a problem, which means that a user can both understand and control parts of the model reasoning process. We demonstrate the feasibility of CSI with a co-designed case study of a document summarization system.

show abstract

Section: Discussionmentioning

confidence: 99%

Visual Interaction with Deep Learning Models through Collaborative Semantic Inference

Gehrmann

Strobelt

Kruger³

et al. 2019

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…The first action of the fairness pipeline is to clearly understand the machine learning process and its consequences to fairness in decision making. The challenge is how to facilitate such an understanding as many practitioners do not fully recognize how every step in the process could potentially lead to biased decision making [21]. To address this, we propose that a fair decision-making tool should take proactive action to help decision makers understand the possible unfairness at each machine learning stage, by providing an overview with a step-bystep workflow to guide users to examine different notions of fairness.…”

Section: Understandmentioning

confidence: 99%

FairSight: Visual Analytics for Fairness in Decision Making

Ahn

Lin

2019

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Fig. 1. We propose a design framework to protect individuals and groups from discrimination in algorithm-assisted decision making. A visual analytic system, FairSight, is implemented based on our proposed framework, to help data scientists and practitioners make fair decisions. The decision is made through ranking individuals who are either members of a protected group (orange bars) or a non-protected group (green bars). (a) The system provides a pipeline to help users understand the possible bias in a machine learning task as a mapping from the input space to the output space. (b) Different notions of fairnessindividual fairness and group fairnessare measured and summarized numerically and visually. For example, the individual fairness is quantified by how pairwise distances between individuals are preserved through the mapping. The group fairness is quantified by the extent to which it leads to fair outcome distribution across groups, with (i) a 2D plot, (ii) a color-coded matrix , and (iii) a ranked-list plot capturing the pattern of potential biases.The system provides diagnostic modules to help (iv) identify and (v) mitigate biases through (c) investigating features before running a model, and (d) leveraging fairness-aware algorithms during and after the training step.Abstract-Data-driven decision making related to individuals has become increasingly pervasive, but the issue concerning the potential discrimination has been raised by recent studies. In response, researchers have made efforts to propose and implement fairness measures and algorithms, but those efforts have not been translated to the real-world practice of data-driven decision making. As such, there is still an urgent need to create a viable tool to facilitate fair decision making. We propose FairSight, a visual analytic system to address this need; it is designed to achieve different notions of fairness in ranking decisions through identifying the required actions -understanding, measuring, diagnosing and mitigating biases -that together lead to fairer decision making. Through a case study and user study, we demonstrate that the proposed visual analytic and diagnostic modules in the system are effective in understanding the fairness-aware decision pipeline and obtaining more fair outcomes.

show abstract

“…There has been development of semi-automated tools to help practitioners detect subgroup biases [2,3,4,7,9,20,21,22]. However, these tools are often designed in isolation from users [18]. With an insufficient understanding of users, they may be divorced from user needs and expectations [19].…”

Section: Background and Related Workmentioning

confidence: 99%

“…Using demographic information for model auditing, however, often raises privacy issues. Coarse-grained demographic information is often less sensitive and could be employed for bias analysis [18].…”

Section: Design Considerationsmentioning

confidence: 99%

See 1 more Smart Citation

Designing Tools for Semi-Automated Detection of Machine Learning Biases: An Interview Study

Law

Malik

et al. 2020

Preprint

View full text Add to dashboard Cite

1 Machine learning models often make predictions that bias against certain subgroups of input data. When undetected, machine learning biases can constitute significant financial and ethical implications. Semi-automated tools that involve humans in the loop could facilitate bias detection. Yet, little is known about the considerations involved in their design. In this paper, we report on an interview study with 11 machine learning practitioners for investigating the needs surrounding semi-automated bias detection tools. Based on the findings, we highlight four considerations in designing to guide system designers who aim to create future tools for bias detection.

show abstract

Improving Fairness in Machine Learning Systems

Cited by 571 publications

References 57 publications

Visual Interaction with Deep Learning Models through Collaborative Semantic Inference

Visual Interaction with Deep Learning Models through Collaborative Semantic Inference

FairSight: Visual Analytics for Fairness in Decision Making

Designing Tools for Semi-Automated Detection of Machine Learning Biases: An Interview Study

Contact Info

Product

Resources

About