Implementation fidelity in computerised assessment of book reading

Abstract

Measuring the implementation fidelity (IF) or integrity of interventions is extremely important, since without it a positive or negative outcome cannot be interpreted. However, IF is actually measured relatively rarely. Direct and indirect methods of measurement have been used in the past, but tend to over-emphasize teacher behaviour. This paper focuses on student behaviour collated through computers -an interesting alternative. It deals with the reading of real books and reading achievement, for which variables a very large amount of computerised data was available -on 852,295 students in 3243 schools. Reading achievement was measured pre-post with STAR Reading, a computerised item-banked adaptive normreferenced test of reading comprehension. IF came from the Accelerated Reader (AR), which measures understanding of independent reading of real books the student has chosen by a quiz. Results showed higher IF was related to higher achievement. Neither IF nor reading achievement related to socio-economic status. Primary (elementary) schools had higher IF and achievement than secondary (high) schools. Females had higher IF and achievement than males. Students of higher reading ability implemented AR at a higher level, but did not gain in reading at a higher level. However, this computerised method of measuring IF with book reading showed limited reliability, no greater than methods emphasising teacher behaviour. In future, IF measures emphasising student response and those emphasising teacher behaviour need to be blended, although the latter will never generate the sample size of the former. This may be true of implementation fidelity in areas other than book reading.

Using Interactive E-Book User Log Variables to Track Reading Processes and Predict Digital Learning Outcomes

“…However, our results do not indicate such hypothesized association between the time students spent on reading story and feedback pages and the digital learning outcomes. In literature, consensus has not yet been reached about the associations between reading time and reading outcomes (Goldhammer et al, 2014;Topping, 2018). Our findings add one more piece of evidence to the argument that there is no direct correlation between the time reading the e-book pages with students' learning and reading outcomes.…”

“…Similarly, Goldhammer et al (2014) examined the associations between time spent reading digital text and reading competency and found that students who had weaker reading skills tended to spend more time reading the text. However, Topping (2018) had contradictory findings such that students who spent longer time in text reading tended to have better reading comprehension outcomes.…”

Implementation fidelity and attainment in computerized practice of mathematics

“…This central scoring enables the collection of large samples of data. The present paper is a companion to previous papers focused on reading (Topping, 2017(Topping, , 2018.…”