Unintentional consequences of artificial intelligence in dermatology for patients with skin of colour

Butt, Sanaa; Butt, Harun; Gnanappiragasam, Dushyanth

doi:10.1111/ced.14726

Cited by 12 publications

(5 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If the models we use are racially biased, this could exacerbate the current racial divide in the US. 14 Similar research shows poor accuracy or racial bias for AI in population health, 3,6,15,16 dermatology, 15 heart failure, 16 opioid use, 17 kidney function, 18 speech recognition, 19 gender classification, 20 and many others. 21 There are very few validations of AI in general.…”

Section: Implementation Of Ai Results Can Be Racially Biasedmentioning

confidence: 93%

“…Diagnostic AI, such as with skin recognition, has also been shown to be biased. 15 It is easy to imagine how an algorithm focused on clinical decision support for a dermatological disease might worsen health disparity. For example, if the prediction algorithm is trained with a predominantly Caucasian population in diagnosing skin cancer, it could lead to poor accuracy in Black or Brown populations.…”

Section: Implementation Of Ai Results Can Be Racially Biasedmentioning

confidence: 99%

“…For example, if the prediction algorithm is trained with a predominantly Caucasian population in diagnosing skin cancer, it could lead to poor accuracy in Black or Brown populations. 15 In this case, skin cancer could be under-identified and undertreated in dark-skinned populations, worsening already existing medical discrepancies.…”

Section: Implementation Of Ai Results Can Be Racially Biasedmentioning

confidence: 99%

See 2 more Smart Citations

Misguided Artificial Intelligence: How Racial Bias is Built Into Clinical Models

Jindal

2022

Journal of Brown Hospital Medicine

View full text Add to dashboard Cite

Artificial Intelligence is being used today to solve a myriad of problems. While there is significant promise that AI can help us address many healthcare issues, there is also concern that health inequities can be exacerbated. This article looks specifically at predictive models in regards to racial bias. Each phase of the model building process including raw data collection and processing, data labelling, and implementation of the model can be subject to racial bias. This article aims to explore some of the ways in which this occurs.

show abstract

Section: Implementation Of Ai Results Can Be Racially Biasedmentioning

confidence: 93%

Section: Implementation Of Ai Results Can Be Racially Biasedmentioning

confidence: 99%

Section: Implementation Of Ai Results Can Be Racially Biasedmentioning

confidence: 99%

See 1 more Smart Citation

Misguided Artificial Intelligence: How Racial Bias is Built Into Clinical Models

Jindal

2022

Journal of Brown Hospital Medicine

View full text Add to dashboard Cite

show abstract

“…10 Although AI-based methods work well in assessing blood-soaked tissues, they are likely to underperform in assessing photographs of anatomical sites, as AI algorithms are known to be biased toward lighter skin tones. 11 There are several limitations to our observations. First, our results are specific to the bleeding scale used and might be different with other bleeding definitions.…”

Section: Discussionmentioning

confidence: 89%

Bleeding assessment following central venous catheter placement, a direct comparison of prospective and retrospective analyses

van Baarle,

van de Weerdt,

Raasveld

et al. 2024

Transfusion

View full text Add to dashboard Cite

BackgroundReported bleeding incidences following central venous catheter (CVC) placement highly depend on methods of bleeding assessment. To determine the direction and magnitude of the bias associated with retrospective data collection, we used data from the PACER randomized controlled trial and a previous retrospective cohort study.Study Design and MethodsA patient‐level comparison of CVC‐related bleeding severity was made among (1) the prospectively collected clinical bleeding assessment of the PACER trial, (2) centralized assessment of CVC insertion site photographs, and (3) retrospective chart review. Interrater reliability for photographic bleeding assessment and retrospective chart review was assessed using Cohen's κ. The magnitude of underreporting of both methods compared to prospective clinical bleeding assessment at different cutoff points of clinically relevant bleeding was assessed using McNemar's test.ResultsInterrater reliability was acceptable for both methods (κ = 0.583 and κ = 0.481 for photographic assessment and retrospective chart review, respectively). Photographic bleeding assessment led to significant underreporting of bleeding complications at all cutoff points. Retrospective chart review led to significant underreporting of minor bleeding complications, with an odds ratio (95% CI) of 0.17 (0.044–0.51) for the cutoff point grade 1 (i.e., self‐limiting or requiring at most 20 min of manual compression) or higher. There was no significant underreporting of major bleeding complications with retrospective chart review.DiscussionCentralized photographic bleeding assessment and retrospective chart review lead to biased bleeding assessment compared to prospective clinical bleeding assessment.

show abstract

“…The (re)production of racism, as well as racist thinking errors through the use of AI technology, is discussed in many places (Butt et al 2021;Langmia 2021;Yen and Hung 2021), but the arguments do not seem to want to be heard. The lack of addressing dominance and difference relations at the level of AI is not without repercussions, as it enables these relations to persist unquestioned and solidify within a novel normative framework.…”

Section: Anomic Results Of Democratically Oriented Normsmentioning

confidence: 99%

Artificial intelligence-related anomies and predictive policing: normative (dis)orders in liberal democracies

Behnam Shad

2023

AI & Soc

View full text Add to dashboard Cite

This article links three rarely considered dimensions related to the implementation of artificial intelligence (AI)-based technologies in the form of predictive policing and discusses them in relation to liberal democratic societies. The three dimensions are the theoretical embedding and the workings of AI within anomic conditions (1), potential normative disorders emerging from them in the form of thinking errors and discriminatory practices (2) as well as the consequences of these disorders on the psychosocial, and emotional level (3). Against this background, AI-induced anomie is conceptualized as a field of tension that refers to a systematic deterioration of democratic norms that are supposed to create ‘normative orders’, but which, when implemented through AI-supported measures, can reproduce existing discriminations, and establish new kinds of discriminatory relations. In future, these AI-based measures have the potential to lead to opposing normative disorders by emerging in the form of false social norms to an equally false Second Nature. They deprive persons involved of the possibility of individual appropriation of social norms and the specific emotional development associated with it.

show abstract

Unintentional consequences of artificial intelligence in dermatology for patients with skin of colour

Cited by 12 publications

References 3 publications

Misguided Artificial Intelligence: How Racial Bias is Built Into Clinical Models

Misguided Artificial Intelligence: How Racial Bias is Built Into Clinical Models

Bleeding assessment following central venous catheter placement, a direct comparison of prospective and retrospective analyses

Artificial intelligence-related anomies and predictive policing: normative (dis)orders in liberal democracies

Contact Info

Product

Resources

About