Feasible and accurate assessment tools developed for non-specialists are needed to scale community-based models of autism evaluation. The purpose of this study was to evaluate use of the Screening Tool for Autism in Toddlers and Young Children (STAT) when used by primary care practitioners ( n = 10) across a statewide system of early diagnosis set within seven United States primary care practices. Specifically, for 130, 14- to 48-month-old children, we examined (1) agreement between STAT classification and Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) outcome and expert diagnosis of autism, (2) characteristics of children with inaccurate STAT classifications, (3) consistency between STAT classification and primary care practitioner diagnosis, and (4) the relationship between STAT/primary care practitioner classification and expert diagnosis. The STAT demonstrated good concurrent validity with the ADOS-2 (77% agreement). Concordance between specialty trained primary care practitioners with expert diagnosis was 80%–87% across three age-based subgroups of children. Children misclassified by the STAT were older, had higher developmental and adaptive skills, and lower autism symptoms. Primary care practitioner diagnosis aligned with STAT categorical risk classification in 86% of cases, and 73% of consistent classifications between STAT and primary care practitioner diagnosis matched expert diagnosis. Overall, the STAT demonstrates good accuracy when used by non-specialists as part of a diagnostic evaluation. Lay summary Specialists conduct autism evaluations using tools that are expensive and difficult to get trained on. Families often wait a long time and travel far to get a diagnosis for their child. To help with this problem, primary care practitioners can be trained to provide evaluations in local communities. However, usable and accurate tools developed for non-specialists are needed. The Screening Tool for Autism in Toddlers and Young Children (STAT) was created for this purpose, but limited research has been done on accuracy of the tool in community primary care. This study tested the STAT when used by primary care practitioners as part of a diagnostic evaluation in 130, 14- to 48-month-old children. We tested (1) STAT agreement with the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2), and diagnosis based on an expert research evaluation, and (2) the relationship between STAT classification, primary care practitioner diagnosis, and expert diagnosis. STAT classification matched the ADOS-2 in 77% of cases and expert diagnosis in 78% of cases. Autistic children incorrectly classified by the STAT were older, had higher developmental and adaptive skills, and fewer autism symptoms. In 86% of cases, the STAT classification agreed with primary care practitioner diagnosis. STAT classification, primary care practitioner diagnosis, and expert diagnosis agreed in 73% of cases. Overall, the STAT shows good accuracy when used by primary care practitioners as part of a community primary care autism evaluation.
