Data preprocessing techniques for classification without discrimination

Kamiran, Faisal; Calders, Toon

doi:10.1007/s10115-011-0463-8

Cited by 935 publications

(799 citation statements)

References 19 publications

(28 reference statements)

Supporting

Mentioning

793

Contrasting

Unclassified

Order By: Relevance

“…Some authors [11,17] study how to prevent data mining from becoming itself a source of discrimination. In this paper, instead we focus on the data mining problem of detecting discrimination in a dataset of historical decision records, and in the rest of this section, we present the most relevant literature.…”

Section: Related Workmentioning

confidence: 99%

“…Furthermore, we observed that, as for when the ordering of the nodes is provided as an input, Algorithm 2 first constrains the current solution for Suppes' conditions (Lines [12][13][14][15][16][17] and then performs the maximum likelihood estimation of the DAG by hill climbing (Lines 19-29). But this time, these steps are iterated along the neighbor orders by an outer hillclimbing procedure (Lines 5-10 and 31-33).…”

Section: Learning the Temporal Ordering Of Variablesmentioning

confidence: 99%

See 1 more Smart Citation

Exposing the probabilistic causal structure of discrimination

Bonchi¹,

Hajian²,

Mishra³

et al. 2017

Int J Data Sci Anal

View full text Add to dashboard Cite

Discrimination discovery from data is an important data mining task, whose goal is to identify patterns of illegal and unethical discriminatory activities against protected-by-law groups, e.g., ethnic minorities. While any legally valid proof of discrimination requires evidence of causality, the state-of-the-art methods are essentially correlation based, albeit, as it is well known, correlation does not imply causation. In this paper, we take a principled causal approach to discrimination detection following Suppes' probabilistic causation theory. In particular, we define a method to extract, from a dataset of historical decision records, the causal structures existing among the attributes in the data. The result is a type of constrained Bayesian network, which we dub Suppes-Bayes causal network (SBCN). Next, we develop a toolkit of methods based on random walks on top of the SBCN, addressing different anti-discrimination legal concepts, such as direct and indirect discrimination, Milano-Bicocca University, Milan, Italy group and individual discrimination, genuine requirement, and favoritism. Our experiments on real-world datasets confirm the inferential power of our approach in all these different tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Learning the Temporal Ordering Of Variablesmentioning

confidence: 99%

Exposing the probabilistic causal structure of discrimination

Bonchi¹,

Hajian²,

Mishra³

et al. 2017

Int J Data Sci Anal

View full text Add to dashboard Cite

show abstract

Section: Discrimination Preventionmentioning

confidence: 99%

“…Data preprocessing methods [1,12,13,17,21,37,43] modify the historic data to remove discriminatory effect according to some discrimination measure before learning a predictive model. For example, in [17] several methods for modifying data were proposed. These methods include Massaging, which changes the labels of some individuals in the dataset to remove discrimination, Reweighting, which assigns weights to individuals to balance the dataset, and Sampling, which changes the sample sizes of different subgroups to make the dataset discrimination-free.…”

Section: Discrimination Preventionmentioning

confidence: 99%

“…These approaches classify discrimination into different types such as group discrimination, individual discrimination, direct and indirect discrimination. Based on that, methods for discrimination prevention have been proposed [1,7,9,[12][13][14][15][17][18][19]21,35,37,43] which either use data preprocessing or algorithm tweaking. However, these works are mainly based on correlation or association-based measures which cannot be used to estimate the causal effect of the protected attributes on the decision.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Anti-discrimination learning: a causal modeling-based framework

Zhang

2017

Int J Data Sci Anal

View full text Add to dashboard Cite

Anti-discrimination learning is an increasingly important task in data mining. Discrimination discovery is the problem of unveiling discriminatory practices by analyzing a dataset of historical decision records, and discrimination prevention aims to remove discrimination by modifying the biased data and/or the predictive algorithms. Discrimination is causal, which means that to prove discrimination one needs to derive a causal relationship rather than an association relationship. Although it is well known that association does not mean causation, the gap between association and causation is not paid enough attention by many researchers. In this paper, we introduce a causal modeling-based framework for anti-discrimination learning. Discrimination is categorized according to two dimensions: direct/indirect and system/group/individual level. Within the causal framework, we introduce a work for discovering and preventing both direct and indirect system-level discrimination in the training data, and a work for extending the non-discrimination result from the training data to prediction. We then introduce two works for group-level direct discrimination and individual-level direct discrimination respectively. The aim of this paper is to deepen the understanding of discrimination in data mining from the causal modeling perspective, and suggest several potential future research directions.

show abstract

Evaluating Bias-Mitigated Predictive Models of Perinatal Mood and Anxiety Disorders

Wong,

Saini,

Accortt

et al. 2024

JAMA Netw Open

View full text Add to dashboard Cite

ImportanceMachine learning for augmented screening of perinatal mood and anxiety disorders (PMADs) requires thorough consideration of clinical biases embedded in electronic health records (EHRs) and rigorous evaluations of model performance.ObjectiveTo mitigate bias in predictive models of PMADs trained on commonly available EHRs.Design, Setting, and ParticipantsThis diagnostic study collected data as part of a quality improvement initiative from 2020 to 2023 at Cedars-Sinai Medical Center in Los Angeles, California. The study inclusion criteria were birthing patients aged 14 to 59 years with live birth records and admission to the postpartum unit or the maternal-fetal care unit after delivery.ExposurePatient-reported race and ethnicity (7 levels) obtained through EHRs.Main Outcomes and MeasuresLogistic regression, random forest, and extreme gradient boosting models were trained to predict 2 binary outcomes: moderate to high-risk (positive) screen assessed using the 9-item Patient Health Questionnaire (PHQ-9), and the Edinburgh Postnatal Depression Scale (EPDS). Each model was fitted with or without reweighing data during preprocessing and evaluated through repeated K-fold cross validation. In every iteration, each model was evaluated on its area under the receiver operating curve (AUROC) and on 2 fairness metrics: demographic parity (DP), and difference in false negatives between races and ethnicities (relative to non-Hispanic White patients).ResultsAmong 19 430 patients in this study, 1402 (7%) identified as African American or Black, 2371 (12%) as Asian American and Pacific Islander; 1842 (10%) as Hispanic White, 10 942 (56.3%) as non-Hispanic White, 606 (3%) as multiple races, 2146 (11%) as other (not further specified), and 121 (&lt;1%) did not provide this information. The mean (SD) age was 34.1 (4.9) years, and all patients identified as female. Racial and ethnic minority patients were significantly more likely than non-Hispanic White patients to screen positive on both the PHQ-9 (odds ratio, 1.47 [95% CI, 1.23-1.77]) and the EPDS (odds ratio, 1.38 [95% CI, 1.20-1.57]). Mean AUROCs ranged from 0.610 to 0.635 without reweighing (baseline), and from 0.602 to 0.622 with reweighing. Baseline models predicted significantly greater prevalence of postpartum depression for patients who were not non-Hispanic White relative to those who were (mean DP, 0.238 [95% CI, 0.231-0.244]; P &lt; .001) and displayed significantly lower false-negative rates (mean difference, −0.184 [95% CI, −0.195 to −0.174]; P &lt; .001). Reweighing significantly reduced differences in DP (mean DP with reweighing, 0.022 [95% CI, 0.017-0.026]; P &lt; .001) and false-negative rates (mean difference with reweighing, 0.018 [95% CI, 0.008-0.028]; P &lt; .001) between racial and ethnic groups.Conclusions and RelevanceIn this diagnostic study of predictive models of postpartum depression, clinical prediction models trained to predict psychometric screening results from commonly available EHRs achieved modest performance and were less likely to widen existing health disparities in PMAD diagnosis and potentially treatment. These findings suggest that is critical for researchers and physicians to consider their model design (eg, desired target and predictor variables) and evaluate model bias to minimize health disparities.

show abstract

Data preprocessing techniques for classification without discrimination

Cited by 935 publications

References 19 publications

Exposing the probabilistic causal structure of discrimination

Exposing the probabilistic causal structure of discrimination

Anti-discrimination learning: a causal modeling-based framework

Evaluating Bias-Mitigated Predictive Models of Perinatal Mood and Anxiety Disorders

Contact Info

Product

Resources

About