Diagnostic Accuracy of Primary Care Clinicians Across a Statewide System of Autism Evaluation

Abstract

OBJECTIVES: To evaluate the diagnostic accuracy of the Early Autism Evaluation (EAE) Hub system, a statewide network that provides specialized training and collaborative support to community primary care providers in the diagnosis of young children at risk for autism spectrum disorder (ASD). METHODS: EAE Hub clinicians referred children, aged 14 to 48 months, to this prospective diagnostic study for blinded follow-up expert evaluation including assessment of developmental level, adaptive behavior, and ASD symptom severity. The primary outcome was agreement on categorical ASD diagnosis between EAE Hub clinician (index diagnosis) and ASD expert (reference standard). RESULTS: Among 126 children (mean age: 2.6 years; 77% male; 14% Latinx; 66% non-Latinx white), 82% (n = 103) had consistent ASD outcomes between the index and reference evaluation. Sensitivity was 81.5%, specificity was 82.4%, positive predictive value was 92.6%, and negative predictive value was 62.2%. There was no difference in accuracy by EAE Hub clinician or site. Across measures of development, there were significant differences between true positive and false negative (FN) cases (all Ps < .001; Cohen’s d = 1.1–1.4), with true positive cases evidencing greater impairment. CONCLUSIONS: Community-based primary care clinicians who receive specialty training can make accurate ASD diagnoses in most cases. Diagnostic disagreements were predominately FN cases in which EAE Hub clinicians had difficulty differentiating ASD and global developmental delay. FN cases were associated with a differential diagnostic and phenotypic profile. This research has significant implications for the development of future population health solutions that address ASD diagnostic delays.

Eye-Tracking Biomarkers and Autism Diagnosis in Primary Care

“…Next, to examine whether individual and composite eye-tracking biomarkers provided additional diagnostic utility beyond that provided by EAE Hub PCP diagnosis and diagnostic certainty, these variables as well as the interaction between diagnosis and certainty were included in logistic regression (with each biomarker in a separate model) to estimate reference standard autism diagnosis. The diagnosis × certainty interaction was included in the model, as previous research has shown that correct classification by PCPs is associated with diagnostic certainty …”

“…This diagnostic study was part of a larger project evaluating the diagnostic accuracy across the Early Autism Evaluation (EAE) Hub system, a statewide network of community PCPs trained to provide streamlined diagnostic assessment of children aged 14 to 48 months referred for autism evaluation. 45 Primary care practitioners in the EAE Hubs follow a standard evaluation protocol and then issue a best-estimate autism diagnosis and report with clinical recommendations (additional details are provided in the eMethods in Supplement 1 ). This study was approved by the Indiana University School of Medicine Institutional Review Board, and written informed consent was obtained from legal guardians of all participants.…”

“…The diagnosis × certainty interaction was included in the model, as previous research has shown that correct classification by PCPs is associated with diagnostic certainty. 45 , 52 …”

Enhancing access to autism diagnostic services for children in the community

“…Penner et al's (2023) recent study on the concordance of autism diagnosis between general pediatricians and a "gold standard" multi-disciplinary team also showed a very high positive predictive value (0.89) for predicting an autism diagnosis in young children (1.5-5.5 years of age). A recent publication from McNally Keehn et al (2023) reported on diagnostic accuracy of primary care providers in Indiana's Early Autism Evaluation Hub and found very similar results (i.e., a positive predictive value of 0.93). In interpreting these results, it is important to note that there are no available data on agreement between expert teams to use as a comparator, as well as no commonly agreed-upon standards for diagnostic accuracy.…”

“…More broadly, we should also understand which general pediatrician assessors are best for this model. Existing studies have relied on pediatricians who were either willing to undergo training (McNally Keehn et al, 2023;Sohl et al, 2023) or who were interested in participating in a study on diagnostic accuracy (Penner et al, 2023). As such, these results cannot be generalized to all pediatricians or primary care physicians; instead, this group could be considered a "coalition of the willing" who might also have a higher baseline knowledge of autism.…”