Bias Discovery in Machine Learning Models for Mental Health

Mosteiro, Pablo; Kuiper, Jesse; Masthoff, Judith; Scheepers, Floortje; Spruit, Marco

doi:10.3390/info13050237

Cited by 12 publications

(6 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our clinicians support the idea that the focus on fairness should be on ensuring that individuals at risk of depression are equally identified across groups, so they are fairly provided with the necessary care and support. This objective aligns with the use of equal opportunity criterion, which is also considered by previous studies in ML for mental health 9 , 30 . In the absence of an expert-informed opinion, the equalized odds criterion 31 serves as an alternative fairness objective which offers a stricter standard than the equal opportunity criterion.…”

Section: Methodsmentioning

confidence: 75%

See 1 more Smart Citation

Fairness and bias correction in machine learning for depression prediction across four study populations

Dang,

Cascarano,

Mulder

et al. 2024

Sci Rep

View full text Add to dashboard Cite

A significant level of stigma and inequality exists in mental healthcare, especially in under-served populations. Inequalities are reflected in the data collected for scientific purposes. When not properly accounted for, machine learning (ML) models learned from data can reinforce these structural inequalities or biases. Here, we present a systematic study of bias in ML models designed to predict depression in four different case studies covering different countries and populations. We find that standard ML approaches regularly present biased behaviors. We also show that mitigation techniques, both standard and our own post-hoc method, can be effective in reducing the level of unfair bias. There is no one best ML model for depression prediction that provides equality of outcomes. This emphasizes the importance of analyzing fairness during model selection and transparent reporting about the impact of debiasing interventions. Finally, we also identify positive habits and open challenges that practitioners could follow to enhance fairness in their models.

show abstract

Section: Methodsmentioning

confidence: 75%

“…Only a handful of studies have adopted methods to counteract bias. For instance, reweighing (RW) bias-mitigation technique 33 was used to minimize bias when forecasting future benzodiazepine administrations 30 . Likewise, others applied Suppression (SUP) 34 and RW approaches to reduce bias in the prediction of postpartum depression 9 .…”

Section: Methodsmentioning

confidence: 99%

Fairness and bias correction in machine learning for depression prediction across four study populations

Dang,

Cascarano,

Mulder

et al. 2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…In the generation of clinical cases, ChatGPT-4 failed to create cases that depicted demographic diversity and relied on stereotypes when choosing gender or ethnicity [27]. Thus, the need for "fair AI" has been pointed out with the goal to develop prediction models that provide equivalent outputs for identical individuals who differ only in one sensitive attribute [28]. To avoid or at least reduce potential bias and move towards fair AI, this bias first needs to be conceptualized, measured, and understood [21].…”

Section: Biases and Responsible Aimentioning

confidence: 99%

Exploring Bias(es) of Large Language Models in the Field of Mental Health – A Comparative Study Investigating the Effect of Gender and Sexual Orientation in Anorexia Nervosa and Bulimia Nervosa Case Vignettes (Preprint)

Schnepper,

Roemmel,

Schaefert

et al. 2024

Preprint

View full text Add to dashboard Cite

BACKGROUND Large language models (LLMs) are increasingly used in the mental health field, with promising results in assessing mental disorders. However, correctness, dependability, and equity of LLM-generated information have been questioned. Amongst other, societal biases and research underrepresentation of certain population strata may affect LLMs. Because LLMs are already used for clinical practice, including decision support, it is important to investigate potential biases to ensure a responsible use of LLMs. OBJECTIVE We aimed to estimate the presence and size of bias related to gender and sexual orientation produced by a common LLM, exemplified in the context of ED symptomatology and health-related quality of life (HRQoL) of patients with AN or BN. METHODS We extracted 30 case vignettes (22 AN, 8 BN) from scientific articles. We adapted each vignette to create 4 versions, describing a female vs. male patient living with their female vs. male partner (2x2 design), yielding n=120 vignettes. We then fed each vignette into Chat Generative Pre-trained Transformer-4 (ChatGPT-4) thrice with the instruction to evaluate them by providing responses to two psychometric instruments, the RAND-36 questionnaire assessing HRQoL and the eating disorder examination questionnaire (EDE-Q). With the resulting LLM-generated scores, we calculated multilevel models (MLMs) with a random intercept for gender and sexual orientation (accounting for within-vignette variance), nested in vignettes (accounting for between-vignette variance). RESULTS The MLM with N=360 observations indicated for the RAND-36 mental composite summary, a significant association with gender (conditional means: 12.8 for male and 15.1 for female cases; 95% CI of the effect=[-6.15, -0.35]; p=.037) but neither with sexual orientation nor an interaction effect (ps>.370). We found no indications for main or interaction effects of gender or sexual orientation for the EDE-Q overall score (conditional means: 5.59-5.65; ps>.611). CONCLUSIONS LLM-generated estimates of mental HRQoL in AN or BN case vignettes are at risk of being affected by cases’ gender, with male cases scoring lower. Given the lack of real-world epidemiological evidence for such a pattern, our study highlights relevant risk of bias when applying generative AI in the context of mental health. Better understanding and mitigation of risk of bias related to gender and other factors, such as ethnicity or socioeconomic status, are highly warranted to ensure responsible use of LLMs when conducting diagnostic assessments or providing treatment recommendations.

show abstract

“…IBM launched AI Fairness 360 [30][31][32], which can help detect and mitigate unwanted bias in machine learning models and datasets. It provides around 70 fairness metrics to test for bias and 11 algorithms to reduce bias in datasets and models, thereby reducing software bias and improving its fairness (e.g., [33]).…”

Section: Status Quomentioning

confidence: 99%

AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications

Chen,

Wu,

Wang

2023

Applied Sciences

View full text Add to dashboard Cite

This article provides a comprehensive overview of the fairness issues in artificial intelligence (AI) systems, delving into its background, definition, and development process. The article explores the fairness problem in AI through practical applications and current advances and focuses on bias analysis and fairness training as key research directions. The paper explains in detail the concept, implementation, characteristics, and use cases of each method. The paper explores strategies to reduce bias and improve fairness in AI systems, reviews challenges and solutions to real-world AI fairness applications, and proposes future research directions. In addition, this study provides an in-depth comparative analysis of the various approaches, utilizing cutting-edge research information to elucidate their different characteristics, strengths, and weaknesses. The results of the comparison provide guidance for future research. The paper concludes with an overview of existing challenges in practical applications and suggests priorities and solutions for future research. The conclusions provide insights for promoting fairness in AI systems. The information reviewed in this paper is drawn from reputable sources, including leading academic journals, prominent conference proceedings, and well-established online repositories dedicated to AI fairness. However, it is important to recognize that research nuances, sample sizes, and contextual factors may create limitations that affect the generalizability of the findings.

show abstract

Bias Discovery in Machine Learning Models for Mental Health

Cited by 12 publications

References 42 publications

Fairness and bias correction in machine learning for depression prediction across four study populations

Fairness and bias correction in machine learning for depression prediction across four study populations

Exploring Bias(es) of Large Language Models in the Field of Mental Health – A Comparative Study Investigating the Effect of Gender and Sexual Orientation in Anorexia Nervosa and Bulimia Nervosa Case Vignettes (Preprint)

AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications

Contact Info

Product

Resources

About