Deflation-Corrected Estimators of Reliability

Abstract

Underestimation of reliability is discussed from the viewpoint of deflation in estimates of reliability caused by artificial systematic technical or mechanical error in the estimates of correlation (MEC). Most traditional estimators of reliability embed product–moment correlation coefficient (PMC) in the form of item–score correlation (Rit) or principal component or factor loading (λi). PMC is known to be severely affected by several sources of deflation such as the difficulty level of the item and discrepancy of the scales of the variables of interest and, hence, the estimates by Rit and λi are always deflated in the settings related to estimating reliability. As a short-cut to deflation-corrected estimators of reliability, this article suggests a procedure where Rit and λi in the estimators of reliability are replaced by alternative estimators of correlation that are less deflated. These estimators are called deflation-corrected estimators of correlation (DCER). Several families of DCERs are proposed and their behavior is studied by using polychoric correlation coefficient, Goodman–Kruskal gamma, and Somers delta as examples of MEC-corrected coefficients of correlation.

Rank–Polyserial Correlation: A Quest for a “Missing” Coefficient of Correlation

“…When it comes to the factual estimation of the inferred association of two observed ordinal or interval-scaled variables with (theoretical, unobservable) latent variables, we have established routines for estimating R PC (e.g., [19][20][21]; see also [22]), as well as R BS and R PS (see [23]). The traditional routines of calculating the estimates by R BS and R PS led, however, to practicalities that the estimates reached out of range values (R BS , R PS >> +1) if the embedded PMC and the item variance are high to start with (e.g., [24]; see the discussion in, e.g., [25,26]; see the computational form in Eq. 34).…”

“…Although the options for R RP discussed here are not restricted to item analysis settings, their characteristics are studied in the framework of measurement modelling. After all, estimators of the association have a central role to play, for example, as the estimators of the item-score association and embedded to estimators of reliability (see discussion in, e.g., [26,29,30]). This perspective leads us to compare the options of R RP with its traditional alternatives: R PP = ρ gX , often called item-total correlation (Rit), Henrysson's corrected item-total correlation (R PPH ) [31], also known as item-rest correlation (Rir), coefficient eta directed so that X explains the order in g or "g given X" 2 , 2 A specific peculiarity in naming of the directions may be necessary to discuss here (see also [2,29,32]) because the directional estimators are used in what follows.…”

“…(originally, in [49], corrected [45]) where D is the observed value of D g |X , and A is as in Eq. (30). Sampling variances and asymptotic standard errors of these estimators are discussed in Metsämuuronen [45] (see also Supplementary Appendix 1).…”

“…It is a well-known fact that PMC is prone both to attenuation caused by errors in measurement modelling and to radical deflation caused by a technical or mechanical errors in the calculation process. These concepts are discussed, amongst others, by Chan [77], Gadermann et al [78], Lavrakas [79], and Metsämuuronen [26,30,80].…”

“…32). To make this radical deflation visible in the measurement model, Metsämuuronen [26,29,30] has proposed a general measurement model, combining a latent variable (θ), observed values of an item (x i ), and a weight factor w iθ that links θ with item i , and the measurement error e iθ :…”