AI pitfalls and what not to do: mitigating bias in AI

Abstract

Various forms of artificial intelligence (AI) applications are being deployed and used in many healthcare systems. As the use of these applications increases, we are learning the failures of these models and how they can perpetuate bias. With these new lessons, we need to prioritize bias evaluation and mitigation for radiology applications; all the while not ignoring the impact of changes in the larger enterprise AI deployment which may have downstream impact on performance of AI models. In this paper, we provide an updated review of known pitfalls causing AI bias and discuss strategies for mitigating these biases within the context of AI deployment in the larger healthcare enterprise. We describe these pitfalls by framing them in the larger AI lifecycle from problem definition, data set selection and curation, model training and deployment emphasizing that bias exists across a spectrum and is a sequela of a combination of both human and machine factors.

Machine Learning and Bias in Medical Imaging: Opportunities and Challenges

“…During training (Phase 2) Use bias mitigation model development strategies 76,77 (eg, data augmentation, adversarial training) Anticipate and evaluate for potential algorithmic bias 59 After training (Phase 3)…”

“…Examine performance overall and across subgroups, ideally in a clinical trial setting if possible Assess fairness metrics (eg, demographic parity, equalized odds or opportunity, disparate impact) with attention to potential effects of demographic proxies (eg, race proxies include patterns of health care utilization) 59 Compare performance with and without including additional variables relevant to the outcome, for example, fusion models including clinical, biomarker, genomic, social determinants of health, and other variables Consider re-training, based on analysis and interpretation of performance metrics Generate model cards for algorithms 78 and data sheets for data sets 79…”

“…[86][87][88][89] It has been noted that employing multiple such methods may or may not improve overall model performance. 59,76,90 Thus, additional strategies include integrating data from alternate sources such as laboratory, biomarker, multiomics, and social determinants of health variables in fusion models. Indeed, multiple studies have shown that unsupervised learning approaches applied to a combination of clinical, genetic, and imaging data can identify novel distinct-and likely more inclusive-phenotypes of disease that can serve to improve predictions of both disease outcomes and their responses to interventions.…”

“…To date, AI technologies have gained approval without necessarily having robust external validation, 96 and the only high-level guidance is provided to promote fairness without any specific requirements to ensure mitigation of potential bias. 59 Notwithstanding plans to establish processes for monitoring of AI model performance in realworld settings, 93 there are growing calls for the FDA to establish a more structured process for initial evaluation and approval of AI technologies that are ideally aligned with how drugs and devices are regulated. 97,98 For this reason, we offer a framework for considering approaches to bias mitigation that are based on existing workflows for building new AI technologies while emphasizing transparency and also approximating the conventional 4 phases of drug development and review (Table 4).…”

“…84 During model training, many bias mitigation strategies can be used in addition to data augmentation approaches (eg, adversarial training). 59,76,77,85 Importantly, such approaches may yet be subject to algorithmic bias and should be used with care. After initial model training, a growing number of metrics are now available to assist with identifying and minimizing bias that may appear for certain subgroups (eg, equal opportunity difference, disparate impact, FNR and FOR difference, general entropy index, statistical parity).…”