Model-based discrimination analysis

Ramadan, Qusai; Ahmadian, Amir Shayan; Strüber, Daniel; Jürjens, Jan; Staab, Steffen

doi:10.1145/3194770.3194775

Cited by 8 publications

(4 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such methods should be integrated into existing data debugging tools such as Dagger [114]. To our knowledge, only two data-focused systems have been implemented towards this goal-by Ruggieri et al [153] for discrimination discovery, by Ramadan et al [147] to uncover unwanted data dependencies through data flow analysis-and a few frameworks (e.g. AIF360 [20], Aequitas [155]) with fairness metrics and algorithmic mitigation methods not integrated into the entire system lifecycle.…”

Section: Framework and Toolsmentioning

confidence: 99%

Managing bias and unfairness in data for decision support: a survey of machine learning and data engineering approaches to identify and mitigate bias and unfairness within data management and analytics systems

2021

View full text Add to dashboard Cite

The increasing use of data-driven decision support systems in industry and governments is accompanied by the discovery of a plethora of bias and unfairness issues in the outputs of these systems. Multiple computer science communities, and especially machine learning, have started to tackle this problem, often developing algorithmic solutions to mitigate biases to obtain fairer outputs. However, one of the core underlying causes for unfairness is bias in training data which is not fully covered by such approaches. Especially, bias in data is not yet a central topic in data engineering and management research. We survey research on bias and unfairness in several computer science domains, distinguishing between data management publications and other domains. This covers the creation of fairness metrics, fairness identification, and mitigation methods, software engineering approaches and biases in crowdsourcing activities. We identify relevant research gaps and show which data management activities could be repurposed to handle biases and which ones might reinforce such biases. In the second part, we argue for a novel data-centered approach overcoming the limitations of current algorithmic-centered methods. This approach focuses on eliciting and enforcing fairness requirements and constraints on data that systems are trained, validated, and used on. We argue for the need to extend database management systems to handle such constraints and mitigation methods. We discuss the associated future research directions regarding algorithms, formalization, modelling, users, and systems.

show abstract

Section: Framework and Toolsmentioning

confidence: 99%

Managing bias and unfairness in data for decision support: a survey of machine learning and data engineering approaches to identify and mitigate bias and unfairness within data management and analytics systems

2021

View full text Add to dashboard Cite

show abstract

“…However, so far there exists no approach that allows modeling fairness requirements and detecting conflicts between them and other data-protection requirements as early as during the design phase of the targeted system. The need for integrating fairness in the early design stages is first has been highlighted by our work in [57,58].…”

Section: Fairness-aware Approachesmentioning

confidence: 99%

A semi-automated BPMN-based framework for detecting conflicts between security, data-minimization, and fairness requirements

et al. 2020

Self Cite

View full text Add to dashboard Cite

Requirements are inherently prone to conflicts. Security, data-minimization, and fairness requirements are no exception. Importantly, undetected conflicts between such requirements can lead to severe effects, including privacy infringement and legal sanctions. Detecting conflicts between security, data-minimization, and fairness requirements is a challenging task, as such conflicts are context-specific and their detection requires a thorough understanding of the underlying business processes. For example, a process may require anonymous execution of a task that writes data into a secure data storage, where the identity of the writer is needed for the purpose of accountability. Moreover, conflicts not arise from trade-offs between requirements elicited from the stakeholders, but also from misinterpretation of elicited requirements while implementing them in business processes, leading to a non-alignment between the data subjects' requirements and their specifications. Both types of conflicts are substantial challenges for conflict detection. To address these challenges, we propose a BPMN-based framework that supports: (i) the design of business processes considering security, data-minimization and fairness requirements, (ii) the encoding of such requirements as reusable, domain-specific patterns, (iii) the checking of alignment between the encoded requirements and annotated BPMN models based on these patterns, and (iv) the detection of conflicts between the specified requirements in the BPMN models based on a catalog of domain-independent anti-patterns. The security requirements were reused from SecBPMN2, a security-oriented BPMN 2.0 extension, while the fairness and data-minimization parts are new. For formulating our patterns and anti-patterns, we extended a graphical query language called SecBPMN2-Q. We report on the feasibility and the usability of our approach based on a case study featuring a healthcare management system, and an experimental user study.

show abstract

“…Deep neural networks (DNNs) [38] have been increasingly adopted in many fields, including computer vision [5], natural language processing [19], software engineering [13,18,32,39,45,48], etc. However, one of the crucial factors hindering DNNs from further serving applications with social impact is the unintended individual discrimination [44,47,55]. Individual discrimination exists when a given instance different from another only in sensitive attributes (e.g., gender, race, etc.)…”

Section: Introductionmentioning

confidence: 99%

NeuronFair: Interpretable White-Box Fairness Testing through Biased Neuron Identification

Zheng¹,

Chen²,

Du³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep neural networks (DNNs) have demonstrated their outperformance in various domains. However, it raises a social concern whether DNNs can produce reliable and fair decisions especially when they are applied to sensitive domains involving valuable resource allocation, such as education, loan, and employment. It is crucial to conduct fairness testing before DNNs are reliably deployed to such sensitive domains, i.e., generating as many instances as possible to uncover fairness violations. However, the existing testing methods are still limited from three aspects: interpretability, performance, and generalizability. To overcome the challenges, we propose NeuronFair, a new DNN fairness testing framework that differs from previous work in several key aspects: (1) interpretable -it quantitatively interprets DNNs' fairness violations for the biased decision; (2) effective -it uses the interpretation results to guide the generation of more diverse instances in less time; (3) generic -it can handle both structured and unstructured data. Extensive evaluations across 7 datasets and the corresponding DNNs demonstrate NeuronFair's superior performance. For instance, on structured datasets, it generates much more instances (∼ ×5.84) and saves more time (with an average speedup of 534.56%) compared with the state-of-the-art methods. Besides, the instances of NeuronFair can also be leveraged to improve the fairness of the biased DNNs, which helps build more fair and trustworthy deep learning systems. The code of NeuronFair is open-sourced at https:// github.com/ haibinzheng/ NeuronFair. CCS CONCEPTS• Computing methodologies → Artificial intelligence; • Software and its engineering → Software reliability.

show abstract

Model-based discrimination analysis

Cited by 8 publications

References 14 publications

Managing bias and unfairness in data for decision support: a survey of machine learning and data engineering approaches to identify and mitigate bias and unfairness within data management and analytics systems

Managing bias and unfairness in data for decision support: a survey of machine learning and data engineering approaches to identify and mitigate bias and unfairness within data management and analytics systems

A semi-automated BPMN-based framework for detecting conflicts between security, data-minimization, and fairness requirements

NeuronFair: Interpretable White-Box Fairness Testing through Biased Neuron Identification

Contact Info

Product

Resources

About