Investigating for bias in healthcare algorithms: a sex-stratified analysis of supervised machine learning models in liver disease prediction

Abstract

ObjectivesThe Indian Liver Patient Dataset (ILPD) is used extensively to create algorithms that predict liver disease. Given the existing research describing demographic inequities in liver disease diagnosis and management, these algorithms require scrutiny for potential biases. We address this overlooked issue by investigating ILPD models for sex bias.MethodsFollowing our literature review of ILPD papers, the models reported in existing studies are recreated and then interrogated for bias. We define four experiments, training on sex-unbalanced/balanced data, with and without feature selection. We build random forests (RFs), support vector machines (SVMs), Gaussian Naïve Bayes and logistic regression (LR) classifiers, running experiments 100 times, reporting average results with SD.ResultsWe reproduce published models achieving accuracies of >70% (LR 71.31% (2.37 SD) – SVM 79.40% (2.50 SD)) and demonstrate a previously unobserved performance disparity. Across all classifiers females suffer from a higher false negative rate (FNR). Presently, RF and LR classifiers are reported as the most effective models, yet in our experiments they demonstrate the greatest FNR disparity (RF; −21.02%; LR; −24.07%).DiscussionWe demonstrate a sex disparity that exists in published ILPD classifiers. In practice, the higher FNR for females would manifest as increased rates of missed diagnosis for female patients and a consequent lack of appropriate care. Our study demonstrates that evaluating biases in the initial stages of machine learning can provide insights into inequalities in current clinical practice, reveal pathophysiological differences between the male and females, and can mitigate the digitisation of inequalities into algorithmic systems.ConclusionOur findings are important to medical data scientists, clinicians and policy-makers involved in the implementation medical artificial intelligence systems. An awareness of the potential biases of these systems is essential in preventing the digital exacerbation of healthcare inequalities.

Sex and gender considerations in Alzheimer’s disease: The Women’s Brain Project contribution

“…Using desirable bias AI can help in more precise and effective diagnostics for females and male ( Stanovsky et al, 2020 ; Castaneda et al, 2022 ). For example, the training of AI algorithms can potentially increase accuracy if sex is considered ( Straw and Wu, 2022 ).…”

Digital health—high tech or high touch?

“…The impact of this topic has been demonstrated by several studies to date. Thoughtful selection of training data and awareness of this issue in clinical application is of absolute necessity [ 27 – 30 ].…”

21st century medicine and emerging biotechnological syndromes: a cross- disciplinary systematic review of novel patient presentations in the age of technology

“…Patient's with higher body habitus may be at increased risk of device failures, as suggested by Katsaras et al who consider the impact of body habitus on loss of telemetric device function in their 63F pacemaker-dependant patient (75). Women and girls face additional risks, particular regarding technology-facilitated abuse and psychiatric misdiagnosis (25,(76)(77)(78)(79).…”