Relationships of Measurement Error and Prediction Error in Observed‐Score Regression

Moses, Tim

doi:10.1111/j.1745-3984.2012.00182.x

Cited by 6 publications

(6 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The regression function's prediction error variance in Equation 5 can be similarly decomposed into parts attributable to true score variance and measurement error variance (Moses, ; Yen & Lu, in preparation),

\begin{matrix} true \\ right σ^{2} ((), Y - \overset{Y}{true} | X) & = & left σ^{2} {(Y - \hat{Y} | X)}_{T X, T Y} \\ left + σ^{2} {(Y - \hat{Y} | X)}_{ɛ X, ɛ Y} . \end{matrix}

…”

Section: Comparisons Of Regression and Scaling Functions’ Prediction mentioning

confidence: 99%

Quantifying Error and Uncertainty Reductions in Scaling Functions: An ITEMS Module

Moses

2014

Educational Measurement

Self Cite

View full text Add to dashboard Cite

This module describes and extends X‐to‐Y regression measures that have been proposed for use in the assessment of X‐to‐Y scaling and equating results. Measures are developed that are similar to those based on prediction error in regression analyses but that are directly suited to interests in scaling and equating evaluations. The regression and scaling function measures are compared in terms of their uncertainty reductions, error variances, and the contribution of true score and measurement error variances to the total error variances. The measures are also demonstrated as applied to an assessment of scaling results for a math test and a reading test. The results of these analyses illustrate the similarity of the regression and scaling measures for scaling situations when the tests have a correlation of at least .80, and also show the extent to which the measures can be adequate summaries of nonlinear regression and nonlinear scaling functions, and of heteroskedastic errors. After reading this module, readers will have a comprehensive understanding of the purposes, uses, and differences of regression and scaling functions.

show abstract

\begin{matrix} true \\ right σ^{2} ((), Y - \overset{Y}{true} | X) & = & left σ^{2} {(Y - \hat{Y} | X)}_{T X, T Y} \\ left + σ^{2} {(Y - \hat{Y} | X)}_{ɛ X, ɛ Y} . \end{matrix}

…”

Section: Comparisons Of Regression and Scaling Functions’ Prediction mentioning

confidence: 99%

Quantifying Error and Uncertainty Reductions in Scaling Functions: An ITEMS Module

Moses

2014

Educational Measurement

Self Cite

View full text Add to dashboard Cite

show abstract

“…While Shang () shows that SIMEX can accommodate multiple error‐prone covariates in linear QR, a future study is needed to determine the optimal number of prior scores to be included in the SGP estimation. Moses () shows that the prediction error of a linear regression is reduced to a greater extent when the covariates are less correlated to each other, which suggests that it may be worthwhile to include test scores from different subjects as covariates. It would be interesting to test this with the SGP approach in a future study as well.…”

Section: Discussionmentioning

confidence: 99%

Covariate Measurement Error Correction for Student Growth Percentiles Using the SIMEX Method

Shang

VanIwaarden

Betebenner

2014

Educational Measurement

View full text Add to dashboard Cite

In this study, we examined the impact of covariate measurement error (ME) on the estimation of quantile regression and student growth percentiles (SGPs), and find that SGPs tend to be overestimated among students with higher prior achievement and underestimated among those with lower prior achievement, a problem we describe as ME endogeneity in this article. We proceeded to assess the effect of covariate ME correction on SGP estimation at two levels-the individual (student) and the aggregate (classroom). Our ME correction approach was limited to the simulation-extrapolation method known as SIMEX. For both the individual and aggregate SGP, we find SIMEX effective in bias reduction. Further, because SIMEX is especially effective in reducing SGP bias for students with very high or very low prior achievement, it significantly weakens the ME endogeneity. SIMEX is also effective in reducing the MSE of aggregate SGP, provided that the students are sorted to some extent on their latent prior achievement. Our empirical study confirms the pattern of the simulation results: SIMEX mainly affects the mean SGP of classes in the highest and lowest quintiles of the prior score distribution, and significantly lowers the correlation between class SGP and prior achievement.

show abstract

“…Equations 25 and 26 suggest that some types of analyses that utilize observed scores to compute correlations and regressions can be inaccurate due to measurement errors of Y, X, or the combination of Y, X, and additional predictor variables (Moses, 2012). Examples of analyses that can be rendered inaccurate when X is unreliable are covariance analyses that match groups based on X (Linn & Werts, 1971b) and differential prediction studies that evaluate X's bias (Linn & Werts, 1971a …”

Section: Scores As Predictorsmentioning

confidence: 99%

Ets Psychometric Contributions: Focus on Test Scores

Moses

2013

ETS Research Report Series

Self Cite

View full text Add to dashboard Cite

Since its 1947 founding, ETS has conducted and disseminated scientific research to support its products and services, and to advance the measurement and education fields. In keeping with these goals, ETS is committed to making its research freely available to the professional community and to the general public. Published accounts of ETS research, including papers in the ETS R&D Scientific and Policy Contributions series, undergo a formal peerreview process by ETS staff to ensure that they meet established scientific and professional standards. All such ETS-conducted peer reviews are in addition to any reviews that outside organizations may provide as part of their own publication processes. Peer review notwithstanding, the positions expressed in the ETS R&D Scientific and Policy Contributions series and other published accounts of ETS research are those of the authors and not necessarily those of the Officers and Trustees of ETS. In 2010, we began to synthesize these scientific and policy contributions, with the intention to release a series of reports sequentially over the course of the next few years. These reports constitute the ETS R&D Scientific and Policy Contributions Series.In the sixth report in the series, Tim Moses examines the contributions to the psychometric study of test scores made by Educational Testing in the past 60 years since the founding of the organization. These contributions can be roughly organized into three areas. The first area, the measurement properties of tests and developments of classical test theory, covers ETS contributions that provide culminating and progressively more rigorous formalizations of classical test theory. These contributions and others by ETS researchers in this area have led to improvements in the psychometric (measurement) quality of tests. The second area is the use of tests as predictors in correlational and regression relationships. ETS contributions in this area have helped to define appropriate and valid uses for tests. The third area is the integration and application of measurement theories and correlational and regression analyses to address testscore issues, which is relevant to operational work on testing programs.This report complements two other upcoming reports on psychometrics in the ETS R&D Scientific and Policy Contributions series, one of which by Neil Dorans will focus on fairness and another by James Carlson that will focus on items.

show abstract

Relationships of Measurement Error and Prediction Error in Observed‐Score Regression

Cited by 6 publications

References 19 publications

Quantifying Error and Uncertainty Reductions in Scaling Functions: An ITEMS Module

Quantifying Error and Uncertainty Reductions in Scaling Functions: An ITEMS Module

Covariate Measurement Error Correction for Student Growth Percentiles Using the SIMEX Method

Ets Psychometric Contributions: Focus on Test Scores

Contact Info

Product

Resources

About