Analysis of Race and Sex Bias in the Autism Diagnostic Observation Schedule (ADOS-2)

Kalb, Luther G.; Singh, Vini; Hong, John J; Holingue, Calliope; Ludwig, Natasha N; Pfeiffer, Danika; Reetzke, Rachel; Gross, Alden L.; Landa, Rebecca

doi:10.1001/jamanetworkopen.2022.9498

Cited by 32 publications

(27 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, multiple studies on MI/DIF have been carried out with different ASD symptom measures, including the Childhood Autism Rating Scale (CARS) (Schopler et al, 1988(Schopler et al, , 2010, ADOS (Lord et al, 2000(Lord et al, , 2012, Social Responsiveness Scale, and ADI-R (Constantino, 2005;Constantino and Gruber, 2012). These studies primarily focused on the effects of race/ethnicity, sex/gender and chronological age on scores (ADOS: Harrison et al, 2017;Ronkin et al, 2021;Burrows et al, 2022;Kalb et al, 2022;CARS: Stevanovic et al, 2021;SRS and ADI-R: Frazier and Hardan, 2017), with a few studies also investigating MI across groups with or without ID (Sturm et al, 2017;Dovgan et al, 2019). While these studies provided preliminary evidence that ASD symptom measures should take the impact of cognitive abilities into account, understanding of how cognitive or language abilities influence the measurement of specific ASD symptoms is still limited.…”

Section: Introductionmentioning

confidence: 99%

Bias in measurement of autism symptoms by spoken language level and non-verbal mental age in minimally verbal children with neurodevelopmental disorders

et al. 2022

View full text Add to dashboard Cite

Increasing numbers of children with known genetic conditions and/or intellectual disability are referred for evaluation of autism spectrum disorder (ASD), highlighting the need to refine autism symptom measures to facilitate differential diagnoses in children with cognitive and language impairments. Previous studies have reported decreased specificity of ASD screening and diagnostic measures in children with intellectual disability. However, little is known about how cognitive and language abilities impact the measurement of specific ASD symptoms in this group. We aggregated a large sample of young children (N = 1196; aged 31–119 months) to examine measurement invariance of ASD symptoms among minimally verbal children within the context of the Autism Diagnostic Observation Schedule (ADOS) Module 1. Using confirmatory factor analysis (CFA) and moderated non-linear factor analysis (MNLFA), we examined how discrete behaviors were differentially associated with the latent symptom domains of social communication impairments (SCI) and restricted and repetitive behaviors (RRB) across spoken language levels and non-verbal mental age groupings. While the two-factor structure of SCI and RRB held consistently across language and cognitive levels, only partial invariance was observed for both ASD symptom domains of SCI and RRB. Specifically, four out of the 15 SCI items and one out of the three RRB items examined showed differential item functioning between children with “Few to No Words” and those with “Some Words”; and one SCI item and one RRB item showed differential item functioning across non-verbal mental age groups. Moreover, even after adjusting for the differential item functioning to reduce measurement bias across groups, there were still differences in ASD symptom domain scores across spoken language levels. These findings further underscore the influence of spoken language level on measurement of ASD symptoms and the importance of measuring ASD symptoms within refined spoken language levels, even among those with minimal verbal abilities.

show abstract

Section: Introductionmentioning

confidence: 99%

Bias in measurement of autism symptoms by spoken language level and non-verbal mental age in minimally verbal children with neurodevelopmental disorders

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Moreover, for all ADOS-2 modules tested, the maximum difference in expected ADOS-2 total scores attributable to measurement bias was less than 1 scale point (range, 0.07-0.91 point). 5 Overall, these findings indicate that the degree of race and sex bias present in the ADOS-2 is low and unlikely to contribute to the systematic underdiagnosis of ASD in Black or female children.A major strength of the study by Kalb et al 5 is its innovative use of item response theory (IRT) models to examine the ADOS-2 items and quantify the degree of bias (or differential item functioning [DIF] in IRT terms) between demographic groups. IRT is a modern, large-sample, psychometric method used to develop, evaluate, and score psychological tests, 7 and IRT models provide mathematical descriptions of how certain item responses (eg, the endorsement of a symptom on the ADOS-2 as 0, 1, or 2) relate to unmeasured latent variables (ie, ASD severity in the case of the ADOS-2) that are assumed to underlie all items on the scale.…”

mentioning

confidence: 70%

mentioning

confidence: 70%

Race and Sex Bias in the Autism Diagnostic Observation Schedule (ADOS-2) and Disparities in Autism Diagnoses

Williams

2022

JAMA Netw Open

View full text Add to dashboard Cite

Autism spectrum disorder (ASD) is a heterogeneous neurodevelopmental condition that affects an estimated 1 in 44 children in the US, with a male-to-female ratio of approximately 4:1. 1 The prevalence of diagnosed ASD has increased substantially over the past 2 decades, although this increase in prevalence has been greater in certain demographic groups, such as female patients 2 and those from minoritized racial ethnic groups, 3 suggesting the presence of diagnostic disparities by race and sex. 4 Notably, the prototypical behavioral manifestations of ASD (on which existing diagnostic criteria and standardized diagnostic instruments are based) were derived from samples of children who were predominantly White and male, 4 and, thus, systematic biases in the diagnostic tools used to evaluate individuals with suspected ASD could theoretically contribute to observed diagnostic disparities.Kalb et al 5 sought to investigate this issue by quantifying the magnitude and practical impact of race-based and sex-based bias in the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2), a clinician-administered measure that is widely used in both research and clinical practice to establish or confirm a diagnosis of ASD. Leveraging ADOS-2 data from 6269 youth attending specialist diagnostic evaluations at a university-based ASD clinic, the authors 5 quantified the degree to which different versions (ie, modules) of the ADOS-2 systematically underestimated the features of ASD in Black/African American children (vs White children) and female children (vs male children).Although their analysis did reveal significant race-based bias in 8 items and sex-based bias in 5 items, 5 estimated effect size metrics 6 indicated that for all but 2 items (D4, Repetitive Interest [racebased bias], and D2, Hand Mannerisms [sex-based bias]), these effects were small and unlikely to be of practical significance. Moreover, for all ADOS-2 modules tested, the maximum difference in expected ADOS-2 total scores attributable to measurement bias was less than 1 scale point (range, 0.07-0.91 point). 5 Overall, these findings indicate that the degree of race and sex bias present in the ADOS-2 is low and unlikely to contribute to the systematic underdiagnosis of ASD in Black or female children.A major strength of the study by Kalb et al 5 is its innovative use of item response theory (IRT) models to examine the ADOS-2 items and quantify the degree of bias (or differential item functioning [DIF] in IRT terms) between demographic groups. IRT is a modern, large-sample, psychometric method used to develop, evaluate, and score psychological tests, 7 and IRT models provide mathematical descriptions of how certain item responses (eg, the endorsement of a symptom on the ADOS-2 as 0, 1, or 2) relate to unmeasured latent variables (ie, ASD severity in the case of the ADOS-2) that are assumed to underlie all items on the scale. Within an IRT framework, researchers are able to test for DIF between groups by examining whether a given item is differentially relate...

show abstract

“…This, in turn, may lead to differences in perception, identification, and scoring of RRBs in standardized assessments, and ultimately later age of diagnosis in females—a commonly observed occurrence (Harrop et al, 2021 ). Indeed, girls are less likely to meet diagnostic thresholds than boys, despite having equally high levels of autistic traits (Dworzynski et al, 2012 ; Kalb et al, 2022 ; Mo et al, 2021 ), and teachers report significantly fewer concerns about social skills in girls than boys (Hiller et al, 2014 ). This suggests that females may not be referred for diagnosis as often, or as early, as males.…”

Section: Discussionmentioning

confidence: 99%

Exclusion of females in autism research: Empirical evidence for a “leaky” recruitment‐to‐research pipeline

D’Mello

Frosch

et al. 2022

Autism Research

View full text Add to dashboard Cite

Autism spectrum disorder (ASD) is characterized by challenges in social communication and the presence of repetitive behaviors or restricted interests. Notably, males are four times as likely as females to be diagnosed with autism. Despite efforts to increase representation and characterization of autistic females, research studies consistently enroll small samples of females, or exclude females altogether. Importantly, researchers often rely on standardized measures to confirm diagnosis prior to enrollment in research studies. We retrospectively analyzed the effects of one such measure (Autism Diagnostic Observation Schedule, ADOS) on research inclusion/exclusion rates by sex in autistic adults, all of whom had a preexisting community diagnosis of autism (n = 145, 95 male, 50 female). Using the ADOS as a confirmatory diagnostic measure resulted in the exclusion of autistic females at a rate over 2.5 times higher than that of autistic males. We compared sex ratios in our sample to those in other large, publically available datasets that rely either on community diagnosis (6 datasets, total n = 42,209) or standardized assessments (2 datasets, total n = 214) to determine eligibility of participants for research. Reliance on community diagnosis rather than confirmatory diagnostic assessments resulted in significantly more equal sex ratios. These results provide evidence for a “leaky” recruitment‐to‐research pipeline for females in autism research. Lay Summary Despite efforts to increase the representation of autistic females in research, studies consistently enroll small samples of females or exclude females altogether. We find that despite making up almost 50% of the initially recruited sample based upon self‐report of community diagnosis, autistic females are disproportonately excluded from research participation as a result of commonly used autism diagnostic measures. In our sample, and several other publically available datasets, reliance on community diagnosis resulted in significantly more equal sex ratios.

show abstract

Analysis of Race and Sex Bias in the Autism Diagnostic Observation Schedule (ADOS-2)

Cited by 32 publications

References 72 publications

Bias in measurement of autism symptoms by spoken language level and non-verbal mental age in minimally verbal children with neurodevelopmental disorders

Bias in measurement of autism symptoms by spoken language level and non-verbal mental age in minimally verbal children with neurodevelopmental disorders

Race and Sex Bias in the Autism Diagnostic Observation Schedule (ADOS-2) and Disparities in Autism Diagnoses

Exclusion of females in autism research: Empirical evidence for a “leaky” recruitment‐to‐research pipeline

Contact Info

Product

Resources

About