Artificial Intelligence Bias in Health Care: Web-Based Survey

Vorisek, Carina Nina; Stellmach, Caroline; Mayer, Paula Josephine; Klopfenstein, Sophie Anne Inès; Bures, Dominik Martin; Diehl, Anke; Henningsen, Maike; Ritter, Kerstin; Thun, Sylvia

doi:10.2196/41089

Cited by 11 publications

(4 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lastly, in additional analysis, we found evidence suggesting that the effect of subject-targeting discrimination could be more pronounced among the non-binary group than among females and males. This aligns with existing evidence of the prevalence of AI bias against gender minorities in society and their reactions to the emerging technology (e.g., Fosch-Villaronga et al, 2021;Vorisek et al, 2023).…”

Section: Discussionsupporting

confidence: 86%

Algorithmic Gender Bias: Investigating Perceptions of Discrimination in Automated Decision-Making

Kim,

Oh,

Lee

2024

Preprint

View full text Add to dashboard Cite

With the widespread use of artificial intelligence and automated decision-making (ADM), concerns are increasing about automated decisions biased against certain social groups, such as women and racial minorities. The public's skepticism and the danger of algorithmic discrimination are widely acknowledged, yet the role of key factors constituting the context of discriminatory situations is underexplored. This study examined people’s perceptions of gender bias in ADM, focusing on three factors influencing the responses to discriminatory automated decisions: the target of discrimination (subject vs. other), the gender identity of the subject, and situational contexts that engender biases. Based on a randomized experiment (N = 602), we found stronger negative reactions to automated decisions that discriminate against the gender group of the subject than those discriminating against other gender groups, evidenced by lower perceived fairness and trust in ADM, and greater negative emotion and tendency to question the outcome. The negative reactions were more pronounced among participants in underserved gender groups than men. Also, participants were more sensitive to biases in economic and occupational contexts than in other situations. These findings suggest that perceptions of algorithmic biases should be understood in relation to the public's lived experience of inequality and injustice in society.

show abstract

Section: Discussionsupporting

confidence: 86%

Algorithmic Gender Bias: Investigating Perceptions of Discrimination in Automated Decision-Making

Kim,

Oh,

Lee

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Currently, AI cannot understand non-verbal cues or body language. Also, bias in data and inaccuracy is troubling [40, 41].…”

Section: Discussionmentioning

confidence: 99%

Large language models for generating medical examinations: systematic review

Artsi,

Sorin,

Konen

et al. 2024

Preprint

View full text Add to dashboard Cite

PurposeWriting multiple choice questions (MCQs) for the purpose of medical exams is challenging. It requires extensive medical knowledge, time and effort from medical educators. This systematic review focuses on the application of large language models (LLMs) in generating medical MCQs.MethodsThe authors searched for studies published up to November 2023. Search terms focused on LLMs generated MCQs for medical examinations. MEDLINE was used as a search database.ResultsOverall, eight studies published between April 2023 and October 2023 were included. Six studies used Chat-GPT 3.5, while two employed GPT 4. Five studies showed that LLMs can produce competent questions valid for medical exams. Three studies used LLMs to write medical questions but did not evaluate the validity of the questions. One study conducted a comparative analysis of different models. One other study compared LLM-generated questions with those written by humans.All studies presented faulty questions that were deemed inappropriate for medical exams. Some questions required additional modifications in order to qualify.ConclusionsLLMs can be used to write MCQs for medical examinations. However, their limitations cannot be ignored. Further study in this field is essential and more conclusive evidence is needed. Until then, LLMs may serve as a supplementary tool for writing medical examinations.

show abstract

“…There is overwhelming evidence that there are biases in various artificial intelligence models (AIMs) applied to machine learning algorithms (MLAGs) used in health care and other industries [ 7 , 16 , 35 , 36 , 37 , 38 , 39 ]. The uses of some of these MLAGs impact and affect many lives and livelihoods, and in many cases, they eventually prove to be devastating to those affected by them [ 22 , 23 , 40 ].…”

Section: The Current State Of Machine Learning (Ml) Models In Health ...mentioning

confidence: 99%

“…Herein, we will be referring to “bias” not as a lack of internal validity or the imprecise gauging of a relationship(s) between a given exposure and an outcome or effect in a population with particular characteristics [ 1 ], although these are important aspects of other types of bias, but rather to describe the problems associated with gathering, generating, processing, training, and evaluating data that might lead to preconceived notions or prejudices and discrimination on the basis of sociodemographic features [ 2 , 3 , 4 , 5 , 6 ]. Specifically, we are presenting bias in AIMs, also known as algorithmic bias, described as a model or MLAG yielding a systematically wrong outcome because of differential considerations of certain informational aspects, such as gender, age, race, ethnicity, and socioeconomic status (SES) contained in datasets [ 7 ]. These learned/training data biases from human input, when heavily and/or blindly relied on health care, perpetuate human-like biases towards these discriminatory informational attributes [ 8 ].…”

Section: Introductionmentioning

confidence: 99%

The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective

Franklin,

Stephens,

Piracha

et al. 2024

Life

View full text Add to dashboard Cite

Artificial intelligence models represented in machine learning algorithms are promising tools for risk assessment used to guide clinical and other health care decisions. Machine learning algorithms, however, may house biases that propagate stereotypes, inequities, and discrimination that contribute to socioeconomic health care disparities. The biases include those related to some sociodemographic characteristics such as race, ethnicity, gender, age, insurance, and socioeconomic status from the use of erroneous electronic health record data. Additionally, there is concern that training data and algorithmic biases in large language models pose potential drawbacks. These biases affect the lives and livelihoods of a significant percentage of the population in the United States and globally. The social and economic consequences of the associated backlash cannot be underestimated. Here, we outline some of the sociodemographic, training data, and algorithmic biases that undermine sound health care risk assessment and medical decision-making that should be addressed in the health care system. We present a perspective and overview of these biases by gender, race, ethnicity, age, historically marginalized communities, algorithmic bias, biased evaluations, implicit bias, selection/sampling bias, socioeconomic status biases, biased data distributions, cultural biases and insurance status bias, conformation bias, information bias and anchoring biases and make recommendations to improve large language model training data, including de-biasing techniques such as counterfactual role-reversed sentences during knowledge distillation, fine-tuning, prefix attachment at training time, the use of toxicity classifiers, retrieval augmented generation and algorithmic modification to mitigate the biases moving forward.

show abstract

Artificial Intelligence Bias in Health Care: Web-Based Survey

Cited by 11 publications

References 44 publications

Algorithmic Gender Bias: Investigating Perceptions of Discrimination in Automated Decision-Making

Algorithmic Gender Bias: Investigating Perceptions of Discrimination in Automated Decision-Making

Large language models for generating medical examinations: systematic review

The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective

Contact Info

Product

Resources

About