Validity Estimates and Functionality of Materials and Procedures Used to Monitor the Implementation Integrity of a Reading Intervention

Begeny, John C.; Upright, James J.; Easton, Julia; Ehrenbock, Cassia; Tunstall, Kali

doi:10.1080/15377903.2013.810187

Cited by 8 publications

(9 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This investigation was the first known study to systematically develop and evaluate a comprehensive set of materials and procedures for training and providing feedback to interventionists using a structured academic intervention. Previous research (Begeny et al., ) demonstrated that the implementation integrity materials and procedures described in this study quickly assisted HELPS interventionists with implementing the program with strong integrity. In this study, we examined two key characteristics of the developed procedures and materials, including their psychometric reliability and their feasibility for learning and using them.…”

Section: Discussionmentioning

confidence: 79%

“…Therefore, the purpose of this two‐part study was to examine (a) the extent to which relatively novice educators could self‐learn and successfully use an implementation integrity monitoring system designed to evaluate a structured reading intervention program, and (b) the inter‐observer reliability of two individuals using the system to evaluate the same interventionist. This two‐part study is also a follow‐up to earlier work showing that our implementation integrity monitoring and feedback system helped interventionists to use the reading program accurately and consistently (Begeny, Upright, Easton, Ehrenbock, & Tunstall, ). However, despite our earlier work showing that the monitoring and feedback procedures produced the intended outcomes (i.e., that it produced the desired interventionist behaviors), that research did not document whether the system and associated materials were feasible to learn or psychometrically reliable across different observers; hence, the focus of this report.…”

mentioning

confidence: 70%

“…This study involved having interventionists implement the HELPS Program with elementary‐aged students at the community center while trained observers (henceforth referred to as Observer 1 and Observer 2) used the aforementioned implementation integrity monitoring procedures and materials to observe, document, and offer feedback about the interventionists’ implementation. All participants in this study had to previously serve as HELPS interventionists (as described in a related study by Begeny et al., ). To become an interventionist in this study, each participant was required to independently follow the self‐training module described in the HELPS Program Teacher's Manual (see Begeny, , starting on p. 19).…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

The Reliability and User-Feasibility of Materials and Procedures for Monitoring the Implementation Integrity of a Reading Intervention

et al. 2014

View full text Add to dashboard Cite

Within the realm of school-based interventions, implementation integrity is important for practical, legal, and ethical purposes. Unfortunately, evidence suggests that proper monitoring of implementation integrity is often absent from both research and practice. School psychology practitioners and researchers have reported that a major barrier to monitoring integrity is a lack of procedural guidance, and currently there is little research that has examined the psychometric reliability of monitoring procedures and materials. Therefore, the purpose of this two-part study was to examine (a) the extent to which relatively novice educators could self-learn and successfully use an implementation integrity monitoring system designed to evaluate a structured reading intervention program, and (b) the inter-observer reliability of two individuals using the system to evaluate the same interventionist. Overall findings suggested that it is feasible for most individuals to learn the implementation integrity monitoring system (and associated materials) and the system can be used reliably across multiple observers. Implications of these findings are discussed, including how the procedures and materials might be adapted for other intervention programs to assist researchers and practitioners with monitoring implementation integrity. C

show abstract

Section: Discussionmentioning

confidence: 79%

mentioning

confidence: 70%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

The Reliability and User-Feasibility of Materials and Procedures for Monitoring the Implementation Integrity of a Reading Intervention

et al. 2014

View full text Add to dashboard Cite

show abstract

“…That is, the final step of program requires that the instructor systematically reviews each step of the implementation sequence (i.e., of the 13 steps summarized on the one-page implementation flowchart) and then records the fidelity on the progress sheet if any of the steps were forgotten or implemented incorrectly. A previous study with this methodology demonstrated that self-recording is a reliable and valid method of assessing the implementation integrity of HELPS program procedures ( Begeny et al, 2013 ). Thus, based on data from each student’s progress sheet, 99% of the total sessions were conducted with 100% fidelity of the core HELPS-PB procedures.…”

Section: Methodsmentioning

confidence: 99%

Translation and cultural adaptation of the HELPS Reading Fluency Program into Brazilian Portuguese: A report of systematic adaptation processes and initial evidence of efficacy

2023

View full text Add to dashboard Cite

IntroductionAcross multiples languages, research demonstrates the important relationship between reading fluency and comprehension. Put simply, a fluent reader has greater attention and memory resources to use higher-order functions in reading, resulting in better comprehension of text. Some reading fluency interventions have shown positive results in improving students’ text reading fluency and comprehension; however, this research has predominantly been conducted with English-speaking students. For instance, until this report, a comprehensive search revealed only one prior study that evaluated an intervention strategy designed to improve students’ reading fluency in Brazilian Portuguese and no prior studies evaluated an intervention program with that population of students.MethodsThe main goals of this two-part project were to (a) systematically translate, culturally adapt, and pilot test the Helping Early Literacy with Practice Strategies (HELPS) reading fluency program for use in Brazilian Portuguese (referred to as, HELPS-PB); and (b) conduct a preliminary quasi-experimental study of the HELPS-PB program with 23 students in grades 3 to 5 who needed a reading fluency intervention.Results and DiscussionThis report documents the processes and successful adaptation of existing English- and Spanish-versions of HELPS into a new HELPS-PB program. It also offers preliminary evidence showing that students receiving HELPS-PB significantly improved their text reading fluency comparted to students in a control group. Implications for research, practice, and the adaptation of reading fluency programs into other languages are discussed.

show abstract

“…A comprehensive set of materials and procedures for observing use of a structured reading program were developed by Begeny, Upright, Easton, Ehrenbock, and Tunstall (2013) and Begeny, Easton, Upright, Tunstall, and Ehrenbock (2014). They related direct observations to teacher self-report and permanent products generated by the intervention.…”

Section: Direct Studiesmentioning

confidence: 99%

Implementation fidelity in computerised assessment of book reading

Topping

2018

Computers & Education

View full text Add to dashboard Cite

Measuring the implementation fidelity (IF) or integrity of interventions is extremely important, since without it a positive or negative outcome cannot be interpreted. However, IF is actually measured relatively rarely. Direct and indirect methods of measurement have been used in the past, but tend to over-emphasize teacher behaviour. This paper focuses on student behaviour collated through computers -an interesting alternative. It deals with the reading of real books and reading achievement, for which variables a very large amount of computerised data was available -on 852,295 students in 3243 schools. Reading achievement was measured pre-post with STAR Reading, a computerised item-banked adaptive normreferenced test of reading comprehension. IF came from the Accelerated Reader (AR), which measures understanding of independent reading of real books the student has chosen by a quiz. Results showed higher IF was related to higher achievement. Neither IF nor reading achievement related to socio-economic status. Primary (elementary) schools had higher IF and achievement than secondary (high) schools. Females had higher IF and achievement than males. Students of higher reading ability implemented AR at a higher level, but did not gain in reading at a higher level. However, this computerised method of measuring IF with book reading showed limited reliability, no greater than methods emphasising teacher behaviour. In future, IF measures emphasising student response and those emphasising teacher behaviour need to be blended, although the latter will never generate the sample size of the former. This may be true of implementation fidelity in areas other than book reading.

show abstract

Validity Estimates and Functionality of Materials and Procedures Used to Monitor the Implementation Integrity of a Reading Intervention

Cited by 8 publications

References 30 publications

The Reliability and User-Feasibility of Materials and Procedures for Monitoring the Implementation Integrity of a Reading Intervention

The Reliability and User-Feasibility of Materials and Procedures for Monitoring the Implementation Integrity of a Reading Intervention

Translation and cultural adaptation of the HELPS Reading Fluency Program into Brazilian Portuguese: A report of systematic adaptation processes and initial evidence of efficacy

Implementation fidelity in computerised assessment of book reading

Contact Info

Product

Resources

About