Acceptability assessment of school psychology interventions from 2005 to 2017

Abstract

Recommendations from multiple professional organizations (e.g., American Psychological Association, Council for Exceptional Children, National Association of School Psychologists) suggest that collection of data on the social validity in practice and research is necessary. The purpose of this study was to systematically review the inclusion of acceptability measurement, which has been one of the most common way to measure social validity, within the intervention literature published across five school psychology journals between 2005 and 2017. Findings suggested just over one third of intervention studies included acceptability assessment. Intervention studies that were delivered individually, targeted behavior skills, and included treatment integrity data were significantly more likely to include acceptability assessment. When acceptability was measured it was typically evaluated one‐time following treatment completion using self‐report tools completed by teachers. Nearly half of studies employed one of seven published tools and the remaining half used researcher‐created measures. The published tools were adapted in a variety of ways and inconsistently reported either item or total scores making it difficult to summarize these data according to intervention target or delivery format. Implications of findings are described.

Assessing children's perceptions of intervention acceptability ratings

“…Although prior acceptability studies have used case vignette methodologies that contrasted individual characteristics, such as gender (Kazdin, 1980; Miller & Kelley, 1992) and socioeconomic status of the raters (Heffer & Kelley, 1987) and intervention characteristics, such as the severity of problem behavior (Reimers et al, 1992), scant research attention has been paid to examining the impact of individual differences on intervention acceptability outcomes within the context of intervention implementation studies. In addition to this area of need and consistent with prior recommendations in the field (Foster & Mash, 1999; Schwartz & Baer, 1991; Silva et al, 2019), future research should also examine the dynamic nature of intervention acceptability, including examining the role of intervention acceptability before, during, and following intervention implementation. Examining whether intervention acceptability outcomes can be used in vivo to make adaptations may have merit, especially for students who are not evidencing improvement even after receiving empirically‐based interventions that have been intensified or modified.…”

“…Although the importance of assessing students' perceptions of school‐based interventions has been advocated (Silva et al, 2019), empirical studies largely focus on static postintervention assessments without considering the dynamic nature of intervention acceptability, including the interplay between preintervention and postintervention acceptability. In addition to contributing to our theoretical conceptualizations of intervention acceptability (Foster & Mash, 1999; Schwartz & Baer, 1991), examining students' intervention acceptability ratings over time may have implications for school‐based intervention efforts (Diller et al, 2013, Losinski et al, 2014; Smith et al, 2013).…”

“…As noted by Silva et al (2019), student intervention acceptability assessments are less likely to be incorporated into school‐based intervention research studies despite long‐standing recommendations to include consumers during intervention planning, implementation, and evaluation (Kazdin, 1977; Wolf, 1978). Assessing students' perceptions of interventions is important because it increases students' engagement in their education and provides critical information that can impact intervention outcomes.…”

“…To date, only a small number of studies dynamically conducted acceptability assessments. As reported in Silva et al (2019) systematic review, only six studies conducted preintervention and postintervention acceptability assessments, and of these six studies, only two were conducted with students. In the first study, Stage et al (2008) assessed the preintervention and postintervention acceptability of interventions derived from multimethod‐multisource functional behavior assessments with three students experiencing behavioral difficulties using a modified version of the Children's Intervention Rating Profile (Bergan & Kratochwill, 1990).…”

“…In addition, it has been argued that assessing intervention acceptability may bridge the gap between research and practice (Losinski et al, 2014), increase dissemination efforts in school settings (Diller et al, 2013; Smith et al, 2013), and be related to the fidelity of intervention implementation (Klingner et al, 2013). However, results of a recent systematic review examining the prevalence of intervention acceptability assessments published in school psychology journals (Silva et al, 2019) indicated that less than half of the intervention studies (40%) included an acceptability assessment. Further, even though most of the published intervention studies were implemented with students, either individually (32%), class‐wide (35%), or in small student groups (5%), intervention acceptability assessments were most frequently conducted with teachers (74%).…”