The effects of baseline estimation on the reliability, validity, and precision of CBM-R growth estimates.

Abstract

This study examined the effect of baseline estimation on the quality of trend estimates derived from Curriculum Based Measurement of Oral Reading (CBM-R) progress monitoring data. The authors used a linear mixed effects regression (LMER) model to simulate progress monitoring data for schedules ranging from 6-20 weeks for datasets with high and low levels of residual variance (poor and good quality datasets respectively). Three observations per day for the first three days of data collection were generated for baseline estimation. As few as one and as many as nine observations were used to calculate baseline. The number of weeks of progress monitoring and the quality of the dataset were highly influential on the reliability, validity, and precision of simulated growth estimates. Results supported the use of using the median of three observations collected on the first day to estimate baseline, particularly when the first observation of that day systematically underestimated student performance. Collecting a large number of observations to estimate baseline does not appear to improve the quality of CBM-R growth estimates.

Decision Rules for Progress Monitoring in Reading: Accuracy during a Large‐Scale Tier II Intervention

“…One potential reason for the stronger observed performance for median‐based rules is that they are a measure of level, which is typically more reliable than estimates of growth (Raudenbush & Bryk, ) and accounts for potentially extreme data points via use of a median. Such an interpretation is consistent with research that showed that baseline estimation for progress monitoring data is improved by use of a median‐based method that accounts for potentially biased first data points (Van Norman, Christ, & Zopluoglu, ). Practically, however, the interpretation of median‐based decision rules has logistical challenges.…”

Multilevel and empirical reliability estimates of learning growth: A simulation study and empirical illustration

“…Perhaps most often researchers use the split-half odd-even method to estimate the reliability of student growth estimates (VanDerHeyden and Burns, 2008;Christ et al, 2013b;Van Norman et al, 2013). This method requires measurement timepoints to be splitted into the odd and even timepoints.…”

“…Hence, increasing the awareness of researchers in the field that this approach to estimate reliability is available and provides useful psychometric information is the main aim of our paper. In accordance with this aim, we sought to complement existing simulation studies in the field of progress monitoring by examining how well multilevel reliability estimation (as an established method for learning progress reliability estimation; National Center on Intensive Intervention, 2014) and empirical reliability estimation work in a range of conditions used in previous simulation studies (Christ et al, 2012(Christ et al, , 2013aVan Norman et al, 2013). Finally, for illustration purposes, we apply the reliability estimation to real data using the quop-L2 test which is used in the context of learning progress assessment in everyday school contexts (Souvignier et al, 2021).…”

“…Hence, issues related to the reliability of progress monitoring slopes such as schedule and duration (i.e., number of occasions per week and overall number of weeks of data collection), or dataset quality (as operationalized by the amount of residual variance in growth models) have been extensively examined in simulation studies (Christ et al, 2012(Christ et al, , 2013aVan Norman et al, 2013). One major dependent variable in such simulation studies is the true reliability of slope estimates (i.e., the squared correlation between estimated slopes and their true values).…”

Enhancing science, practice, and policy relevant to school psychology around the world.

“…Volume 28, No, 3, contained articles regarding preservice teacher attitudes toward gay and lesbian parents (Herbstrith et al,, 2013); a latent profile analysis of teacher perceptions of parent involvement (Stormont et al, 2013); the development and initial validation of the social and academic behavior risk screener for elementary grades (Kilgus, Chafouleas, & Riley-Tillman, 2013); understanding daily Stressors in school-age children (Escobar, 2013); the effects of baseline estimation on the reliability, validity, and precision of curriculum-based measurement in reading growth estimates (Van Norman, Christ, & Zopluoglu, 2013); and the invariance of WJ-III scores for students with and without learning disorders (Benson & Taub, 2013), Volume 28, No. 4, consisted of articles concerning; the special topic section focused on the current science and practice regarding the assessment of general education teachers' Tier 1 classrooms (Reddy, Fabiano, & Jimerson, 2013), the development and initial validation of the Prekindergarten Classroom Observation tool and goal setting system for data-based coaching (Crawford et al, 2013), initial predictive validity of the Classroom Strategies Scale-observer form-on statewide testing scores , development and construct validity of the Classroom Strategies Scale-observer form , and the role of classroom observation systems in moving toward a shared understanding of effective teaching (Connor, 2013).…”