Fair Adversarial Gradient Tree Boosting

Abstract

Fair classification has become an important topic in machine learning research. While most bias mitigation strategies focus on neural networks, we noticed a lack of work on fair classifiers based on decision trees even though they have proven very efficient. In an up-to-date comparison of state-ofthe-art classification algorithms in tabular data, tree boosting outperforms deep learning [1]. For this reason, we have developed a novel approach of adversarial gradient tree boosting. The objective of the algorithm is to predict the output Y with gradient tree boosting while minimizing the ability of an adversarial neural network to predict the sensitive attribute S. The approach incorporates at each iteration the gradient of the neural network directly in the gradient tree boosting. We empirically assess our approach on 4 popular data sets and compare against state-ofthe-art algorithms. The results show that our algorithm achieves a higher accuracy while obtaining the same level of fairness, as measured using a set of different common fairness definitions.

Learning Optimal Fair Classification Trees

“…In the online setting, Zhang et al [39] propose a fairness-aware Hoeffding tree that builds decision trees over streams of data. Grari et al [21] take an adversarial approach to training fair gradient-boosted trees that promote statistical parity. Kanamori and Arimura [29] propose a post-processing step that uses MIO to edit the branching thresholds of the tree's internal nodes to satisfy some fairness constraint; in the paper they use statistical parity and equalized odds.…”

“…The UCI Adult dataset contains around 45,000 entries from a 1994 Census database [15]. The goal is to predict whether or not someone's income exceeds $50,000 per year, and we treat sex as the sensitive attribute in accordance with [1], [21], and [28], among others -females being the marginalized group.…”

FairXGBoost: Fairness-aware Classification in XGBoost

“…Pre-processing methods [5] [14] typically project the data into a feature space with fair representations. In-processing methods [11][16] [20] involve changing the training procedure in order to make the model predictions fair. Post-processing [12][15] methods typically transform the model outputs to ensure fairness.…”

“…Literature on fairness in ensemble models, especially in boosted tree models is rather limited. To the best of our knowledge, only [9] and [11] consider fairness in a boosting setup. [9] were the first to perform a case study of fairness for Adaboost.…”

“…Regulatory bodies all around the world such as European Union through its General Data Protection and Regulation (GDPR) [1] have been keen on developing ways to make machine learning models fair, accountable, transparent and explainable. A host of bias-mitigation approaches [5] [11][12] [16] have been proposed in recent times that ensure that models are not discriminatory against a specific population. While most of the initial work involved ensuring fairness in the training data being used to train these models, these methods often suffered a loss in model performance.…”