Evaluation and Mitigation of Racial Bias in Clinical Machine Learning Models: Scoping Review (Preprint)

Abstract

BACKGROUND Racial bias is a key concern regarding the development, validation, and implementation of machine learning (ML) models in clinical settings. Despite the potential of bias to propagate health disparities, racial bias in clinical ML has yet to be thoroughly examined and best practices for bias mitigation remain unclear. OBJECTIVE Our objective was to perform a scoping review to characterize the methods by which racial bias of ML has been assessed and describe strategies that may be used to enhance algorithmic fairness in clinical ML. METHODS A scoping review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses Extension for Scoping Reviews. Literature search using the PubMed, Scopus, and Embase databases as well as Google Scholar identified 635 records, of which 12 studies were included. RESULTS Applications of ML were varied and involved diagnosis, outcome prediction, and clinical score prediction performed on datasets including images, diagnostic studies, clinical text, and clinical variables. One study (8%) described a model in routine clinical use, two (17%) examined prospectively validated clinical models, and the remaining nine (75%) described internally validated models. Eight studies (75%) concluded that racial bias was present, two (17%) concluded that it was not, and two (17%) assessed the implementation of bias mitigation strategies without comparison to a baseline model. Fairness metrics used to assess algorithmic racial bias were inconsistent. The most commonly observed metrics were: equal opportunity difference (5/12, 42%); accuracy (4/12, 25%); and disparate impact (2/12, 17%). All eight studies (67%) which implemented methods for mitigation of racial bias successfully increased fairness as measured by the authors’ chosen metrics. Pre-processing methods of bias mitigation were the most commonly used across all studies which implemented them. CONCLUSIONS The broad scope of medical ML applications and potential patient harms demand an increased emphasis on evaluation and mitigation of racial bias in clinical ML. However, the adoption of algorithmic fairness principles in medicine remains inconsistent and is limited by poor data availability and ML model reporting. We recommend that researchers and journal editors emphasize standardized reporting and data availability in medical ML studies to improve transparency and facilitate evaluation for racial bias.

A Review and Tutorial on Machine Learning-Enabled Radar-Based Biomedical Monitoring

“…However, this has not been investigated sufficiently so far due to the lack of large and diverse datasets that are necessary to develop models that are generalizable across the whole population. The creation of diverse datasets is essential to tackle challenges such as gender and race bias in the application of radar-based ML applications [150]. Furthermore, the collection of largescale datasets is challenging due to the characteristics of radar signals.…”