Fair Machine Learning in Healthcare: A Review

Feng, Qizhang; Du, Mengnan; Zou, Na; Hu, Xia

doi:10.48550/arxiv.2206.14397

Cited by 8 publications

(8 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet, evidence for fairness in the PI setting is lacking. Closer to PI, fairness research in the healthcare setting is still in its infancy [40]. The digitization of medical data has enabled the scientific community to collect large amounts of heterogeneous, multi-modal data and develop machine learning algorithms for a variety of medical tasks.…”

Section: Related Workmentioning

confidence: 99%

Uncovering Bias in Personal Informatics

Yfantidou¹,

Sermpezis²,

Vakali³

et al. 2023

Preprint

View full text Add to dashboard Cite

Personal informatics (PI) systems, powered by smartphones and wearables, enable people to lead healthier lifestyles by providing meaningful and actionable insights that break down barriers between users and their health information. Today, such systems are used by billions of users for monitoring not only physical activity and sleep but also vital signs and women's and heart health, among others. Despite their widespread usage, the processing of sensitive PI data may suffer from biases, which may entail practical and ethical implications. In this work, we present the first comprehensive empirical and analytical study of bias in PI systems, including biases in raw data and in the entire machine learning life cycle. We use the most detailed framework to date for exploring the different sources of bias and find that biases exist both in the data generation and the model learning and implementation streams. According to our results, the most affected minority groups are users with health issues, such as diabetes, joint issues, and hypertension, and female users, whose data biases are propagated or even amplified by learning models, while intersectional biases can also be observed. CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing; • Applied computing → Consumer health; • Computing methodologies → Artificial intelligence; • Social and professional topics → Codes of ethics.

show abstract

Section: Related Workmentioning

confidence: 99%

Uncovering Bias in Personal Informatics

Yfantidou¹,

Sermpezis²,

Vakali³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In contrast, an algorithm is unfair if its decisions are skewed toward a particular group of the population without being explained by clinical needs [18,33]. On the basis of the definition of fairness [34], we used 2 metrics to assess fairness-equal opportunity difference (EOD) and disparate impact (DI). Textbox 1 shows the terminology used to define the fairness metrics in this study.…”

Section: Fairness Metricsmentioning

confidence: 99%

Architectural Design of a Blockchain-Enabled, Federated Learning Platform for Algorithmic Fairness in Predictive Health Care: Design Science Study

Liang,

Zhao,

Chen

et al. 2023

J Med Internet Res

View full text Add to dashboard Cite

Background Developing effective and generalizable predictive models is critical for disease prediction and clinical decision-making, often requiring diverse samples to mitigate population bias and address algorithmic fairness. However, a major challenge is to retrieve learning models across multiple institutions without bringing in local biases and inequity, while preserving individual patients’ privacy at each site. Objective This study aims to understand the issues of bias and fairness in the machine learning process used in the predictive health care domain. We proposed a software architecture that integrates federated learning and blockchain to improve fairness, while maintaining acceptable prediction accuracy and minimizing overhead costs. Methods We improved existing federated learning platforms by integrating blockchain through an iterative design approach. We used the design science research method, which involves 2 design cycles (federated learning for bias mitigation and decentralized architecture). The design involves a bias-mitigation process within the blockchain-empowered federated learning framework based on a novel architecture. Under this architecture, multiple medical institutions can jointly train predictive models using their privacy-protected data effectively and efficiently and ultimately achieve fairness in decision-making in the health care domain. Results We designed and implemented our solution using the Aplos smart contract, microservices, Rahasak blockchain, and Apache Cassandra–based distributed storage. By conducting 20,000 local model training iterations and 1000 federated model training iterations across 5 simulated medical centers as peers in the Rahasak blockchain network, we demonstrated how our solution with an improved fairness mechanism can enhance the accuracy of predictive diagnosis. Conclusions Our study identified the technical challenges of prediction biases faced by existing predictive models in the health care domain. To overcome these challenges, we presented an innovative design solution using federated learning and blockchain, along with the adoption of a unique distributed architecture for a fairness-aware system. We have illustrated how this design can address privacy, security, prediction accuracy, and scalability challenges, ultimately improving fairness and equity in the predictive health care domain.

show abstract

“…The suggestions provided, summarized from the literature to date, are not exhaustive. Because fair-aware AI is a relatively new field, standards are still evolving; however, the evidence validating the use of these methods is growing rapidly and has already accumulated an impressive empirical base (e.g., Feng, 2022;Kamishima et al, 2012;Mehrabi et al, 2022;Oneto et al, 2019;Pfohl et al, 2021;Ustun, 2019;Zemel et al, 2013;Zhao et al, 2018). Although a complete review of all possible mitigation methods is outside the scope of this article, we provide a general overview here to guide development and implementation within psychology.…”

Section: Assessing and Mitigating Bias In Aimentioning

confidence: 99%

“…If bias is detected, apply model in-processing and decision postprocessing methods (e.g., Feng, 2022;Kamishima et al, 2012;Mehrabi et al, 2022;Oneto et al, 2019;Pfohl et al, 2021;Ustun, 2019;Zemel et al, 2013;Zhao et al, 2018). Repeat Model Evaluation Steps 1-3 until bias is removed from the model.…”

Section: Bias Mitigationmentioning

confidence: 99%

“…Although research in the fair-aware AI domain is still evolving, early and promising work shows that bias evaluation and mitigation methods are effective in reducing bias in AI algorithms (e.g., Feng, 2022;Hardt et al, 2016;Kamishima et al, 2012;Mehrabi et al, 2022;Oneto et al, 2019;Pfohl et al, 2021;Ustun, 2019;Zemel et al, 2013;Zhao et al, 2018). For example, Ustun et al (2019) used a recursive procedure to decouple subgroups of individuals and train decoupled classifiers for each subgroup.…”

Section: Bias Mitigationmentioning

confidence: 99%

See 1 more Smart Citation

A Call to Action on Assessing and Mitigating Bias in Artificial Intelligence Applications for Mental Health

Timmons

Duong

Fiallo

et al. 2022

Perspect Psychol Sci

View full text Add to dashboard Cite

Advances in computer science and data-analytic methods are driving a new era in mental health research and application. Artificial intelligence (AI) technologies hold the potential to enhance the assessment, diagnosis, and treatment of people experiencing mental health problems and to increase the reach and impact of mental health care. However, AI applications will not mitigate mental health disparities if they are built from historical data that reflect underlying social biases and inequities. AI models biased against sensitive classes could reinforce and even perpetuate existing inequities if these models create legacies that differentially impact who is diagnosed and treated, and how effectively. The current article reviews the health-equity implications of applying AI to mental health problems, outlines state-of-the-art methods for assessing and mitigating algorithmic bias, and presents a call to action to guide the development of fair-aware AI in psychological science.

show abstract

Fair Machine Learning in Healthcare: A Review

Cited by 8 publications

References 61 publications

Uncovering Bias in Personal Informatics

Uncovering Bias in Personal Informatics

Architectural Design of a Blockchain-Enabled, Federated Learning Platform for Algorithmic Fairness in Predictive Health Care: Design Science Study

A Call to Action on Assessing and Mitigating Bias in Artificial Intelligence Applications for Mental Health

Contact Info

Product

Resources

About