Black box fairness testing of machine learning models

Aggarwal, Aniya; Lohia, Pranay; Nagar, Seema; Dey, Kuntal; Saha, Diptikalyan

doi:10.1145/3338906.3338937

Cited by 148 publications

(176 citation statements)

References 12 publications

Supporting

Mentioning

174

Contrasting

Unclassified

Order By: Relevance

“…Besides coverage criteria, A large body of testing methods was proposed for testing machine learning models, such as fuzzing [18,25,44,62,63,68,71,80], symbolic execution [3,23,51,55], runtime validation [54,64], fairness testing [3,62,77], etc. DeepTest [59] utilizes nine types of realistic image transformations for generating test images, which discovered more than 1,000 erroneous behaviors of DNNs used in autonomous driving systems.…”

Section: Related Workmentioning

confidence: 99%

Correlations between deep neural network model coverage criteria and model quality

Yan

Tao

Liu

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

Correlations between deep neural network model coverage criteria and model quality

Yan

Tao

Liu

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

“…Intuitively, "model fairness" is highly correlated to attributes of people. In fact, this design choice is consistently taken by the majority, if not all, of previous research in this field [Galhotra et al, 2017;Udeshi et al, 2018;Aggarwal et al, 2019].…”

Section: Sentence Perturbator Pmentioning

confidence: 99%

“…Existing research has proposed various techniques to expose fairness violations in NLP models. However, these works suffer from various drawbacks, such as relying on heavy-weight (unscalable) statistical or symbolic analysis tools, using pre-defined templates for input generation, or processing only structured data tables [Galhotra et al, 2017;Udeshi et al, 2018;Aggarwal et al, 2019]. Inspired by relevant research in software engineering, we advocate for a focus on formulating the analysis of model fairness as a specific software testing task where typical NLP models are treated as a "black-box."…”

Section: Biased Reviewmentioning

confidence: 99%

“…To date, mutation testing techniques are proposed to detect discrimination in NLP models [Galhotra et al, 2017;Udeshi et al, 2018]. [Aggarwal et al, 2019] introduces symbolic execution and local explainability to generate inputs in black-box scenarios. Nevertheless, existing research essentially uses pre-defined mapping rules (e.g., "male" → "female") to perturb structured data tables (e.g., the Kaggle Bank Marketing Dataset [Martinez, 2018]).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Metamorphic Testing and Certified Mitigation of Fairness Violations in NLP Models

Wang

Liu

2020

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Natural language processing (NLP) models have been increasingly used in sensitive application domains including credit scoring, insurance, and loan assessment. Hence, it is critical to know that the decisions made by NLP models are free of unfair bias toward certain subpopulation groups. In this paper, we propose a novel framework employing metamorphic testing, a well-established software testing scheme, to test NLP models and find discriminatory inputs that provoke fairness violations. Furthermore, inspired by recent breakthroughs in the certified robustness of machine learning, we formulate NLP model fairness in a practical setting as (ε, k)-fairness and accordingly smooth the model predictions to mitigate fairness violations. We demonstrate our technique using popular (commercial) NLP models, and successfully flag thousands of discriminatory inputs that can cause fairness violations. We further enhance the evaluated models by adding certified fairness guarantee at a modest cost.

show abstract

“…For all these reasons, this paper explores ML bias mitigation. In the recent software engineering literature, we have found some works to identify bias in machine learning software systems [7,11]. But there is no prior work done to explain the reason behind the bias and also removing the bias from the software.…”

Section: Introductionmentioning

confidence: 99%

Fairway: a way to build fair ML software

Chakraborty

Majumder

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

106

View full text Add to dashboard Cite

Machine learning software is increasingly being used to make decisions that affect people's lives. But sometimes, the core part of this software (the learned model), behaves in a biased manner that gives undue advantages to a specific group of people (where those groups are determined by sex, race, etc.). This łalgorithmic discriminationž in the AI software systems has become a matter of serious concern in the machine learning and software engineering community. There have been works done to find łalgorithmic biasž or łethical biasž in software system. Once the bias is detected in the AI software system, mitigation of bias is extremely important. In this work, we a) explain how ground truth bias in training data affects machine learning model fairness and how to find that bias in AI software, b) propose a method Fairway which combines preprocessing and in-processing approach to remove ethical bias from training data and trained model. Our results show that we can find bias and mitigate bias in a learned model, without much damaging the predictive performance of that model. We propose that (1) testing for bias and (2) bias mitigation should be a routine part of the machine learning software development life cycle. Fairway offers much support for these two purposes. CCS CONCEPTS • Software and its engineering → Software creation and management; • Computing methodologies → Machine learning.

show abstract

Black box fairness testing of machine learning models

Cited by 148 publications

References 12 publications

Correlations between deep neural network model coverage criteria and model quality

Correlations between deep neural network model coverage criteria and model quality

Metamorphic Testing and Certified Mitigation of Fairness Violations in NLP Models

Fairway: a way to build fair ML software

Contact Info

Product

Resources

About