Search-based fairness testing for regression-based machine learning systems

Abstract

Context Machine learning (ML) software systems are permeating many aspects of our life, such as healthcare, transportation, banking, and recruitment. These systems are trained with data that is often biased, resulting in biased behaviour. To address this issue, fairness testing approaches have been proposed to test ML systems for fairness, which predominantly focus on assessing classification-based ML systems. These methods are not applicable to regression-based systems, for example, they do not quantify the magnitude of the disparity in predicted outcomes, which we identify as important in the context of regression-based ML systems. Method: We conduct this study as design science research. We identify the problem instance in the context of emergency department (ED) wait-time prediction. In this paper, we develop an effective and efficient fairness testing approach to evaluate the fairness of regression-based ML systems. We propose fairness degree, which is a new fairness measure for regression-based ML systems, and a novel search-based fairness testing (SBFT) approach for testing regression-based machine learning systems. We apply the proposed solutions to ED wait-time prediction software. Results: We experimentally evaluate the effectiveness and efficiency of the proposed approach with ML systems trained on real observational data from the healthcare domain. We demonstrate that SBFT significantly outperforms existing fairness testing approaches, with up to 111% and 190% increase in effectiveness and efficiency of SBFT compared to the best performing existing approaches. Conclusion: These findings indicate that our novel fairness measure and the new approach for fairness testing of regression-based ML systems can identify the degree of fairness in predictions, which can help software teams to make data-informed decisions about whether such software systems are ready to deploy. The scientific knowledge gained from our work can be phrased as a technological rule; to measure the fairness of the regression-based ML systems in the context of emergency department wait-time prediction use fairness degree and search-based techniques to approximate it.

Studying the Practices of Deploying Machine Learning Projects on Docker

“…However, the conclusion drawn from such analysis should be supported by further analysis. For example, we have shown in this study that concluding that such features are biased by a simple perturbation of given feature values when other variables are kept unchanged can be misleading (such as [5,17,61]). A genuine conclusion should consider the effect of confounding.…”

“…Perera, Anjana, et al [61] proposed a novel search-based fairness testing (SBFT) approach based on the concept of fairness degree to test for fairness and evaluate the fairness of regression-based ML systems. Their proposed SBFT approach works by computing the maximum difference in the predicted values by the machine learning system for all pairs of identical instances apart from the sensitive features to describe the worst-case behavior of the system.…”

“…However, this also implies that our feature ranking of bias-inducing and feature importance might change if the correlated features were included in the studied dataset. Despite this preprocessing step, it is worth pointing out that the similarly sets of features of the respective dataset has been used in the previous studies [17,80,18,44,61] on bias assessment and mitigation in machine learning. Moreover, in the future we plan to perform a thorough empirical analysis to understand how our process behaves given different types of models and the presence of the correlation.…”

“…In the recent years, the researchers have been proposing methods such as [5,17,61,41,33,63,21] to test if a model is making a fairer decision (and mitigating unfair decision). For instance, enforcing that the true positive or false positive rates, or both [33] are similar for different populations.…”

Fairness Testing: A Comprehensive Survey and Analysis of Trends

“…To tackle this problem, Udeshi et al [78] used a threshold to determine the metamorphic relations, i.e., the outcome difference of two similar instances that differ in the sensitive attribute needs to be smaller than the manually-specified threshold. Similarly, Perera et al [115] proposed the concept of fairness degree, which is measured as the maximum difference in the predicted values for all pairs of instances that are similar apart from the sensitive attribute. The fairness degree can be used and specified to construct metamorphic relations and guide the test input generation.…”