Gender Bias in Artificial Intelligence: Severity Prediction at an Early Stage of COVID-19

Abstract

Artificial intelligence (AI) technologies have been applied in various medical domains to predict patient outcomes with high accuracy. As AI becomes more widely adopted, the problem of model bias is increasingly apparent. In this study, we investigate the model bias that can occur when training a model using datasets for only one particular gender and aim to present new insights into the bias issue. For the investigation, we considered an AI model that predicts severity at an early stage based on the medical records of coronavirus disease (COVID-19) patients. For 5,601 confirmed COVID-19 patients, we used 37 medical records, namely, basic patient information, physical index, initial examination findings, clinical findings, comorbidity diseases, and general blood test results at an early stage. To investigate the gender-based AI model bias, we trained and evaluated two separate models—one that was trained using only the male group, and the other using only the female group. When the model trained by the male-group data was applied to the female testing data, the overall accuracy decreased—sensitivity from 0.93 to 0.86, specificity from 0.92 to 0.86, accuracy from 0.92 to 0.86, balanced accuracy from 0.93 to 0.86, and area under the curve (AUC) from 0.97 to 0.94. Similarly, when the model trained by the female-group data was applied to the male testing data, once again, the overall accuracy decreased—sensitivity from 0.97 to 0.90, specificity from 0.96 to 0.91, accuracy from 0.96 to 0.91, balanced accuracy from 0.96 to 0.90, and AUC from 0.97 to 0.95. Furthermore, when we evaluated each gender-dependent model with the test data from the same gender used for training, the resultant accuracy was also lower than that from the unbiased model.

What Goes In, Must Come Out: Generative Artificial Intelligence Does Not Present Algorithmic Bias Across Race and Gender in Medical Residency Specialties

“…Despite efforts to address gender biases, one study spanning two decades in academic medical centers showed that little progress has been made in transforming predominantly male-dominated fields, and women physicians continue to face significant challenges in these domains [ 21 , 22 ]. One study explored the potential model bias that emerges when training AI models exclusively on one gender dataset for predicting early-stage severity based on COVID-19 patient medical records [ 23 ]. They found that gender-dependent AI models exhibited lower accuracy compared to unbiased models, underscoring the importance of training models on unbiased data.…”

Artificial Intelligence–Driven Respiratory Distress Syndrome Prediction for Very Low Birth Weight Infants: Korean Multicenter Prospective Cohort Study

“…Therefore, our developed model may be biased toward the gestational week range of 26 to 32 weeks. Indeed, our recent study showed that data skewed toward 1 gender can also lead to model bias [ 48 ]. Thus, in the future work, we need to update our model based on a more balanced gestational week data.…”

A Roadmap to Artificial Intelligence (AI): Methods for Designing and Building AI ready Data for Women’s Health Studies

“…Algorithmic bias can introduce inequity in clinical decision making and healthcare research 1,12,16 . To prevent bias and produce reliable results, the data that are the foundation for the algorithms need careful curation especially in women's health 17,20 . This implies data readiness for AI is not a one size fits all solution 2,3 .…”