Racial underrepresentation in dermatological datasets leads to biased machine learning models and inequitable healthcare

Kleinberg, Giona; Diaz, Michael; Batchu, Sai; Lucke‐Wold, Brandon

doi:10.46439/biomedres.3.025

Cited by 5 publications

(3 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work we show how protein Training data bias Biases in training data are reflected in downstream models. Under-represented subgroups can suffer lower accuracy due to insufficient weight in the training data (Buolamwini & Gebru, 2018;Chen et al, 2018;Kleinberg et al, 2022;Shahbazi et al, 2023), and socially undesirable biases in data are often amplified by models (Bolukbasi et al, 2016;Caliskan et al, 2017;Taori & Hashimoto, 2023). Various papers have studied how re-weighting or curating datasets can mitigate these biases (Zhao et al, 2017;Ryu et al, 2017;Tschandl et al, 2018;Yang et al, 2020), even finding that overall performance is improved by over-weighting minority groups and actively increasing diversity in datasets (Gao et al, 2020;Rolf et al, 2021;Lee et al, 2022).…”

Section: Related Workmentioning

confidence: 99%

Protein language models are biased by unequal sequence sampling across the tree of life

Ding,

Steinhardt

2024

Preprint

View full text Add to dashboard Cite

Protein language models (pLMs) trained on large protein sequence databases have been used to understand disease and design novel proteins. In design tasks, the likelihood of a protein sequence under a pLM is often used as a proxy for protein fitness, so it is critical to understand what signals likelihoods capture. In this work we find that pLM likelihoods unintentionally encode a species bias: likelihoods of protein sequences from certain species are systematically higher, independent of the protein in question. We quantify this bias and show that it arises in large part because of unequal species representation in popular protein sequence databases. We further show that the bias can be detrimental for some protein design applications, such as enhancing thermostability. These results highlight the importance of understanding and curating pLM training data to mitigate biases and improve protein design capabilities in under-explored parts of sequence space.

show abstract

Section: Related Workmentioning

confidence: 99%

Protein language models are biased by unequal sequence sampling across the tree of life

Ding,

Steinhardt

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…To mitigate these risks and select the most suitable solution, we usually carry out an accurate a-priori analysis considering factors such as the specific characteristics of the dataset (i.e., data types), the proportion of missing data and the underlying mechanisms causing the missingness [35]. Indeed, missing data can be classified into three different categories depending on the missing data mechanism.…”

Section: Addressing Missing Datamentioning

confidence: 99%

High-Fidelity Synthetic Data Applications for Data Augmentation

Wang,

Draghi,

Rotalinti

et al. 2024

Artificial Intelligence

View full text Add to dashboard Cite

The use of high-fidelity synthetic data for data augmentation is an area of growing interest in data science. In this chapter, the concept of synthetic data is introduced, and different types of synthetic data are discussed in terms of their utility or fidelity. Approaches to synthetic data generation are presented and compared with computer modelling and simulation approaches, highlighting the unique benefits of high-fidelity synthetic data. One of the main applications of high-fidelity synthetic data is supporting the training and validation of machine learning algorithms, where it can provide a virtually unlimited amount of diverse and high-quality data to improve the accuracy and robustness of models. Furthermore, high-fidelity synthetic data can address missing data and biases due to under-sampling using techniques such as BayesBoost, as well as boost sample sizes in scenarios where the real data is based on a small sample. Another important application is generating virtual patient cohorts, such as digital twins, to estimate counterfactuals in silico trials, allowing for better prediction of treatment outcomes and personalised medicine. The chapter concludes by identifying areas for further research in the field, including developing more efficient and accurate synthetic data generation methods and exploring the ethical implications of using synthetic data.

show abstract

“…According to the “2022 Cancer Facts & Data,” the United States has over 5 million new cases of skin cancer annually, with the melanoma incidence rate for white non-Hispanic people being 30 times that of other ethnicities. Nonetheless, much current academic research and challenges from the ISIC database mainly focus on the identification of benign and malignant skin tumors (Kleinberg et al, 2022 ), which means they do not satisfy the requirements of the Chinese ethnic group. Secondly, China and other countries experience a high incidence of skin diseases.…”

Section: Introductionmentioning

confidence: 99%

Deep skin diseases diagnostic system with Dual-channel Image and Extracted Text

Li,

Zhang,

Wei

et al. 2023

Front. Artif. Intell.

View full text Add to dashboard Cite

BackgroundDue to the lower reliability of laboratory tests, skin diseases are more suitable for diagnosis with AI models. There are limited AI dermatology diagnostic models combining images and text; few of these are for Asian populations, and few cover the most common types of diseases.MethodsLeveraging a dataset sourced from Asia comprising over 200,000 images and 220,000 medical records, we explored a deep learning-based system for Dual-channel images and extracted text for the diagnosis of skin diseases model DIET-AI to diagnose 31 skin diseases, which covers the majority of common skin diseases. From 1 September to 1 December 2021, we prospectively collected images from 6,043 cases and medical records from 15 hospitals in seven provinces in China. Then the performance of DIET-AI was compared with that of six doctors of different seniorities in the clinical dataset.ResultsThe average performance of DIET-AI in 31 diseases was not less than that of all the doctors of different seniorities. By comparing the area under the curve, sensitivity, and specificity, we demonstrate that the DIET-AI model is effective in clinical scenarios. In addition, medical records affect the performance of DIET-AI and physicians to varying degrees.ConclusionThis is the largest dermatological dataset for the Chinese demographic. For the first time, we built a Dual-channel image classification model on a non-cancer dermatitis dataset with both images and medical records and achieved comparable diagnostic performance to senior doctors about common skin diseases. It provides references for exploring the feasibility and performance evaluation of DIET-AI in clinical use afterward.

show abstract

Racial underrepresentation in dermatological datasets leads to biased machine learning models and inequitable healthcare

Cited by 5 publications

References 42 publications

Protein language models are biased by unequal sequence sampling across the tree of life

Protein language models are biased by unequal sequence sampling across the tree of life

High-Fidelity Synthetic Data Applications for Data Augmentation

Deep skin diseases diagnostic system with Dual-channel Image and Extracted Text

Contact Info

Product

Resources

About