Measuring parent strategy use in early intervention: Reliability and validity of the Naturalistic Developmental Behavioral Intervention Fidelity Rating Scale across strategy types

Abstract

Children with autism spectrum disorder benefit from early, intensive interventions to improve social communication, and parent-implemented interventions are a feasible, family-centered way to increase treatment dosage. The success of such interventions is dependent on a parent’s ability to implement the strategies with fidelity. However, measurement of parent strategy use varies across studies. Most studies use one of two types of observational coding measures (macro- and micro-codes). Macro-codes are known for being efficient while micro-codes are known for being precise. This study evaluates the reliability and validity of the NDBI-Fi, a macro-code, compared to a micro-code. Parent–child interaction videos for 177 participants were used to compare these measures. Results demonstrated that the NDBI-Fi had strong inter-rater reliability. It also had strong convergent validity with the micro-code after intervention. In addition, the NDBI-Fi was sensitive to change, and it demonstrated precision comparable to the micro-code. Furthermore, a novel scoring procedure detected differences in parents who learned different intervention strategy types. However, the NDBI-Fi did not demonstrate strong validity before intervention, particularly when measuring responsive intervention strategies. Taken together, findings support the use of the NDBI-Fi as an outcome measure, and future work should focus on continued development of valid pre-intervention macro-codes. Lay abstract Children with autism spectrum disorder benefit from early intervention to improve social communication, and parent-implemented interventions are a feasible and family-centered way to increase the amount of treatment they receive. For these treatments to be effective, it is important for the parent to implement the strategies as intended. However, measurement of parent strategy use is inconsistent across studies of parent-implemented interventions. This study evaluates the quality of the NDBI-Fi, an efficient measure, compared to a more time-consuming measure that is known to be precise. Videos of parents playing with their children were used to compare these two measurement methods. Results demonstrated that the NDBI-Fi was of good quality: scorers had high levels of agreement, the NDBI-Fi was similar to the more precise measure in rating parents after intervention, it detected changes from before to after intervention, and it detected differences when parents learned different types of intervention strategies. The NDBI-Fi was not as precise as the other measure across all strategies before parents learned intervention. Taken together, the findings of this study support the use of the NDBI-Fi as a high-quality outcome measure.

Comorbid problems in children with autism spectrum disorder as measured with the Child Behavior Checklist (CBCL) and the effect of the Essential 5® systemic behavioral intervention treatment.

“…Anxiety in children was reduced at a 3-month follow-up after a parent-mediated treatment [20]. Importantly, a systemic approach centres around the transactional model of behavioral development [21], whereby parent's and child's behavior in uence each other continuously [22]. Another strength of a systemic treatment approach is that parental sensitivity improves and the feeling of control in problematic situations with children with ASD increases [14,[23][24][25].…”

The Role of Naturalistic Developmental Behavioral Interventions in Early Intervention for Autistic Toddlers: An Observational Study

“…The NDBI-Fi (Frost et al, 2020;Sone et al, 2021) was used to characterize the fidelity with which SLPs implemented individual NDBI strategies during the recorded sessions. The NDBI-Fi is an observational rating scale consisting of nine items rated on a five-point ordinal scale, with six items measuring developmental strategies (e.g., modeling appropriate language) and three items related to behavioral strategies (e.g., communication temptations; see Table 2 for a full list of strategies included on the NDBI-Fi).…”

“…The NDBI-Fi is an observational rating scale consisting of nine items rated on a five-point ordinal scale, with six items measuring developmental strategies (e.g., modeling appropriate language) and three items related to behavioral strategies (e.g., communication temptations; see Table 2 for a full list of strategies included on the NDBI-Fi). The NDBI-Fi has been validated to measure NDBI strategy use during interactions between caregivers and their autistic child (Frost et al, 2020;Sone et al, 2021), and was slightly adapted to capture SLP strategy use during EI sessions (i.e., accounting for the sometimes-triadic nature of EI sessions, such as when caregivers and SLPs jointly interacted with the child). Guidelines were also created to isolate segments of the entire recorded session (mean session length: 60.0 minutes, SD: 7.6) that were appropriate for coding (i.e., when the SLP actively interacted with the child).…”

“…Coders included two doctoral student speech-language pathologists and one clinical master's student in speech-language pathology. Training consisted of reviewing the coding manual and rating a standard set of videos to reliability criteria established in previous studies (Sone et al, 2021). These criteria included rating three consecutive segments with: 1) seven items rated within one point of the primary codes, 2) no items rated greater than two points apart, and 3) overall mean scores within 0.5 points.…”

“…The SLP pauses between conversational turns to allow the child an opportunity to communicate, waiting at least three seconds before taking another verbal turn. NDBI-Fi=Naturalistic Developmental Behavioral Intervention-Fidelity Rating Scale (Frost et al, 2021) *Item added in Sone et al, 2021…”