Assessing Fidelity of Implementation of an Unprescribed, Diagnostic Mathematics Intervention

Effectiveness of Structured Teacher Adaptations to an Evidence‐Based Summer Literacy Program

“…Thus, core components were operationalized in order to increase teacher autonomy over program activities and processes, while at the same time preserving the underlying program logic. In operationalizing core program components for Adaptive READS teachers, we considered the evidence underlying each component, including both theory and empirical evidence tying particular components to outcomes, to determine which aspects of each component were musts and could not be adapted (Century, Rudnick, & Freeman, 2010;Munter et al, 2014;Waltz, Addis, Koerner, & Jacobson, 1993). For example, prior READS studies provided strong evidence for the effect of scaffolded teacher support on student outcomes (Kim & White, 2008).…”

“…As with fidelity, adaptation has not been consistently defined in the research literature. Some researchers have considered any change to the original program model to be an adaptation, regardless of the extent to which the change adheres to foundational program principles (Domitrovich et al., ; Munter, Wilhelm, Cobb, & Cordray, ; Sherin & Drake, ). Others have argued that clearer lines need to be drawn between fidelity and adaptation, as fidelity may moderate the effect of local adaptation on program outcomes (O'Donnell, ).…”

Supplemental Material for A Power Analysis for Fidelity Measurement Sample Size Determination

“…There are clearly many other possible designs for both the study and fidelity data collection. Munter et al (2014) describe a study in which students were assigned to one of 18 tutors (clusters), but fidelity measurement was made at the student level. Neither of our power analyses would apply to their fidelity sample design because it differed from those we considered.…”

“…In addition, Munter et al (2014) also documented that the uncertainty in their measurements of fidelity were due not only to sampling, but also due to imperfect reliability of the fidelity measurements made by coders. Guidance on reliability requirements could also be addressed by using the objective of adequate power as a guide.…”

“…If a randomized trial of a promising intervention shows that an effect is small, or even nonsignificant, one possible explanation is that its important components were not fully implemented. The importance and complexity of assessing fidelity has been emphasized in recent years, with increasingly sophisticated definitions of fidelity (Dane & Schneider, 1998;Durlak & Dupre, 2008;O'Donnell, 2008) and recommended procedures for developing fidelity instruments and collecting fidelity data (Munter, Wilhelm, Cobb, & Cordray, 2014;Nelson, Cordray, Hulleman, Darrow, & Sommer, 2012). Researchers are urged to be careful and complete when measuring both causes (i.e.…”