Robust Regression for Slope Estimation in Curriculum-Based Measurement Progress Monitoring

Abstract

Although ordinary least-squares (OLS) regression has been identified as a preferred method to calculate rates of improvement for individual students during curriculum-based measurement (CBM) progress monitoring, OLS slope estimates are sensitive to the presence of extreme values. Robust estimators have been developed that are less biased by extreme values; however, the performance of robust estimators in the short data streams typical of CBM progress monitoring is unknown. The purpose of the current study was to investigate bias and efficiency relative to OLS for several robust slope estimators on simulated CBM progress monitoring data. Data were generated at several combinations of series lengths (i.e., 7, 12, and 24 data points) and percentages of extreme value contamination (i.e., 0%, 15%, and 30% of data points). Results indicated that the robust slope estimates were substantially more efficient than OLS in the presence of extreme values. Potential uses of robust slope estimates for calculating students' rates of improvement in CBM progress monitoring are discussed.

A Comparison of Common and Novel Curriculum-Based Measurement of Reading Decision Rules to Predict Spring Performance for Students Receiving Supplemental Interventions

“…The discussant proposed that although they personally initially advocated against the use of data point rules, it may be time to explore ways in which level can be used for decisions. Indeed, substantial effort has been placed in refining trend estimation methods (e.g., Mercer, Lyons, Johnston, & Millhoff, 2014; Vannest, Parker, Davis, Soares, & Smith, 2012). While the findings of this study are not without limitations (see last section), they provide initial support for the notion of improving measures of level for the purposes of decision making.…”

Novice Interpretations of Progress Monitoring Graphs: Extreme Values and Graphical Aids

“…A variety of statistical methods have been used to estimate the trend of student data; until recently, there was evidence that ordinary least squares regression (OLSR) was the most accurate (Shinn, Good, & Stein, 1989). Although older trend estimation techniques may be replaced by new trend estimation techniques as they are disseminated (e.g., robust regression; Mercer, Lyons, Johnston, & Millhoff, 2015), the most common trend calculation of lines displayed on graphs today remains OLSR. This includes trend calculation within frequently used software systems such as AIMSweb, DIBELS, easyCBM, and FAST.…”

Validity Evidence for Progress Monitoring With Star Reading: Slope Estimates, Administration Frequency, and Number of Data Points

“…Maximum Likelihood is another widely used estimator, although it has not been examined in the context of progress monitoring. Theil-Sen and Huber-M are the two slope estimators that are robust to the presence of outliers (e.g., Mercer et al, 2014). We excluded the other robust estimators-SMDM and Tukey's bisquare-from the simulation study because Mercer's, Lyons, Johnston and Millhoff (2014) study has already indicated that Huber-M can outperform the SMDM and Tukey's bisquare estimators when outliers are present in the progress monitoring data.…”

“…Most of the literature has focused on hand-fit trend lines (e.g., based on visual estimations) and linear regression methods, such as ordinary lest-squares (OLS) regression (Ardoin et al, 2013). More recently, the performance of various robust estimators, such as Huber M-estimator, Tukey's bisquare, and SMDM-estimation (i.e., an initial S-estimate, followed by an M-estimate, a Design Adaptive Scale estimate, and a final M-step; see Koller and Stahel, 2011 for more details), has been investigated in the context of CBM and progress monitoring measures (e.g., Mercer et al, 2014). Another robust estimation method, the Theil-Sen estimator, has been proposed to obtain more robust slope estimates from progress monitoring data (e.g., Vannest et al, 2012).…”