Practical, epistemic and normative implications of algorithmic bias in healthcare artificial intelligence: a qualitative study of multidisciplinary expert perspectives

Abstract

BackgroundThere is a growing concern about artificial intelligence (AI) applications in healthcare that can disadvantage already under-represented and marginalised groups (eg, based on gender or race).ObjectivesOur objectives are to canvas the range of strategies stakeholders endorse in attempting to mitigate algorithmic bias, and to consider the ethical question of responsibility for algorithmic bias.MethodologyThe study involves in-depth, semistructured interviews with healthcare workers, screening programme managers, consumer health representatives, regulators, data scientists and developers.ResultsFindings reveal considerable divergent views on three key issues. First, views on whether bias is a problem in healthcare AI varied, with most participants agreeing bias is a problem (which we call the bias-critical view), a small number believing the opposite (the bias-denial view), and some arguing that the benefits of AI outweigh any harms or wrongs arising from the bias problem (the bias-apologist view). Second, there was a disagreement on the strategies to mitigate bias, and who is responsible for such strategies. Finally, there were divergent views on whether to include or exclude sociocultural identifiers (eg, race, ethnicity or gender-diverse identities) in the development of AI as a way to mitigate bias.Conclusion/significanceBased on the views of participants, we set out responses that stakeholders might pursue, including greater interdisciplinary collaboration, tailored stakeholder engagement activities, empirical studies to understand algorithmic bias and strategies to modify dominant approaches in AI development such as the use of participatory methods, and increased diversity and inclusion in research teams and research participant recruitment and selection.

How should artificial intelligence be used in Australian health care? Recommendations from a citizens’ jury

“…They also emphasised rigorous evaluation and fairness, aspects that may be neglected by commercial producers of health care AI. Reported breakthroughs in health care machine learning have often not been supported by more methodologically rigorous scrutiny, 19 and evaluations of health care AI have often focused on overall accuracy rather than bias or fairness 13 . The jury's recommendations suggest that a well informed public might reject these approaches as unjustifiable.…”

“…The four speakers directly answered final questions at the end of the first face‐to‐face day. On the second day, a world café‐style session 12 helped jurors discuss and record their insights about the benefits, harms, and bias and fairness of AI in health care 13 . Jurors then developed a list of questions that might require recommendations, which the research team sorted into draft categories; the entire jury finalised the category list together.…”

The Impact of Generative Conversational Artificial Intelligence on the Lesbian, Gay, Bisexual, Transgender, and Queer Community: Scoping Review

“…Technological advancements can sometimes exacerbate existing inequalities. LGBTQ individuals with limited access to technology or digital literacy might be left behind in terms of benefiting from positive AI impacts [ 51 - 53 ].…”

Promises and Perils of Big Data: Philosophical Constraints on Chemical Ontologies

“…The use of big chemical data and data-driven approaches, however, comes with risks. The problems of biased training data producing biased ML algorithms and, more generally, analysis of biased data producing biased results are well documented across the natural and social sciences. − Misleading results will not only confuse and distract scientific progress but also undermine public confidence in science itself. − To manage difficulties associated with big data, data scientists have developed several metrics for assessing the reliability and accessibility of results from big data use. The “five Vs” − velocity, volume, value, variety and veracityserve as use principles for generating reliable results from big data (Figure ).…”