Validity and Reliability Evidence for Assessments Based in Applied Behavior Analysis: A Systematic Review

Abstract

The current article presents the findings from a systematic review of the available reliability and validity evidence supporting the use of criterion-referenced assessments based on the applied behavior analysis framework. We identified 46 studies that reported reliability and/or validity evidence for six assessments, 37 of which presented reliability evidence and 43 presented validity evidence. Additionally, we extracted and summarized information related to participant characteristics (e.g., age, sex, diagnosis), geographic location, and research setting (e.g., residential facility, home). Overall, we found conflicting support for the use of the assessments. When coupled with the reported usage by behavior analysis professionals, our findings suggest a misalignment between the reportedly used assessments and the number of published studies providing validity and/or reliability evidence. We found inconsistent use of measurement-related vocabulary and that many studies could have been strengthened by conducting different statistical analyses. We provide a summary of studies, findings, and offer recommendations for clinical practice and future measurement research.

An Internal Consistency Reliability Study of the Catalyst Datafinch Applied Behavior Analysis Data Collection Application With Autistic Individuals

“…This could lead to misinterpretation of data, ineffective interventions, and missed opportunities for skill development. Therefore, ongoing psychometric research is essential [17][18][19].…”

“…Scientific discourse calls for the significant need for psychometric studies that delineate the reliability and validity of instruments designed to measure behaviors and skill acquisition in individuals with autism [ 17 ]. Reliability is a crucial aspect of any psychological measurement instrument [ 18 ].…”

“…They ensure that the tool consistently measures what it intends to measure. In the context of autism, this is particularly important due to the wide range of behaviors and skills that can be present in this population [ 17 , 18 ]. Psychometric studies can help improve assessment.…”