The effect of various simultaneous sources of mechanical error in the estimators of correlation causing deflation in reliability: seeking the best options of correlation for deflation-corrected reliability

Abstract

Estimates of reliability by traditional estimators are deflated, because the item-total or item-score correlation (Rit) or principal component or factor loading (λi) embedded in the estimators are seriously deflated. Different optional estimators of correlation that can replace Rit and λi are compared in this article. Simulations show that estimators such as polychoric correlation (RPC), gamma (G), dimension-corrected G (G2), and attenuation-corrected Rit (RAC) and eta (EAC) reflect the true correlation without any loss of information with several sources of technical or mechanical error in the estimators of correlation (MEC) including extreme item difficulty and item variance, small number of categories in the item and in the score, and the varying distributions of the latent variable. To obtain deflation-corrected reliability, RPC, G, G2, RAC, and EAC are likely to be the best options closely followed by r-bireg or r-polyreg coefficient (RREG).

Rank–Polyserial Correlation: A Quest for a “Missing” Coefficient of Correlation

“…By using the concept "discordant pairs, " the last refers to the situation in which all the pairs of observations are discordant. The interpretation of the magnitude of the estimates by JT gX is the same as that in R PP (ρ gX ), with the note that, in real-life datasets, R PP cannot reach perfect +1 or −1 (see e.g., [29,45]) while JT gX can.…”

“…One of the best alternatives for R BS and R PS , if not the best, is a coefficient called r-bireg and rpolyreg correlation (R REG ; see [27,28]; see Eq. 37), which have behaved quite optimally in simulations (see, e.g., [29]).…”

“…Although the options for R RP discussed here are not restricted to item analysis settings, their characteristics are studied in the framework of measurement modelling. After all, estimators of the association have a central role to play, for example, as the estimators of the item-score association and embedded to estimators of reliability (see discussion in, e.g., [26,29,30]). This perspective leads us to compare the options of R RP with its traditional alternatives: R PP = ρ gX , often called item-total correlation (Rit), Henrysson's corrected item-total correlation (R PPH ) [31], also known as item-rest correlation (Rir), coefficient eta directed so that X explains the order in g or "g given X" 2 , 2 A specific peculiarity in naming of the directions may be necessary to discuss here (see also [2,29,32]) because the directional estimators are used in what follows.…”

“…After all, estimators of the association have a central role to play, for example, as the estimators of the item-score association and embedded to estimators of reliability (see discussion in, e.g., [26,29,30]). This perspective leads us to compare the options of R RP with its traditional alternatives: R PP = ρ gX , often called item-total correlation (Rit), Henrysson's corrected item-total correlation (R PPH ) [31], also known as item-rest correlation (Rir), coefficient eta directed so that X explains the order in g or "g given X" 2 , 2 A specific peculiarity in naming of the directions may be necessary to discuss here (see also [2,29,32]) because the directional estimators are used in what follows. With the truly directional estimators D and eta, in widely used software packages as IBM SPSS, SAS, and R-libraries, this specific direction is traditionally named as "X dependent" (see, e.g., [15,[33][34][35][36][37]).…”

“…Inherited from G and D, G 2 and D 2 are based on probability, but, because of the third-order element A, they are described as semi-trigonometric in nature (see [29]). Coefficients G 2 and D 2 can be used both with binary and polytomous items, and, when D = G = 1 and with binary items, G 2 = G and D 2 = D. In simulations [29,45], G 2 tends to give estimates with a magnitude of close to those by R PC . Of G 2 and D 2 , D 2 gives more conservative estimates.…”