Congeneric and (Essentially) Tau-Equivalent Estimates of Score Reliability

Graham, James M.

doi:10.1177/0013164406288165

Cited by 585 publications

(547 citation statements)

References 15 publications

Supporting

Mentioning

499

Contrasting

Unclassified

Order By: Relevance

“…Figure 2 illustrates how the reliability of mean RTs depends on the population variability in the individual cognitive processing and sensory-motor residual time parameters, G, Δ, and R. Somewhat arbitrarily, these levels of population variability were chosen to range from very small valuessimulating homogeneous populations-to values large enough to yield visible effects on reliability. Reliability increases with variability in all three of these parameters, consistent with the well-known phenomenon that the reliability of any measure depends not only on the measuring instrument itself, but also on the population to which it is applied (e.g., Graham, 2006). In particular, the general rule within classical test theory is that reliability increases with the amount of true score variance (Eq.…”

Section: Reliability Of Mean Reaction Timessupporting

confidence: 68%

Mental chronometry and individual differences: Modeling reliabilities and correlations of reaction time means and effect sizes

2013

View full text Add to dashboard Cite

We used a general stage-based model of reaction time (RT) to investigate the psychometric properties of mean RTs and experimental effect sizes (i.e., differences in mean RTs). Using the model, formulas were derived for the reliabilities of mean RTs and RT difference scores, and these formulas provide guidance about the number of trials per participant needed to obtain reliable estimates of these measures. In addition, formulas were derived for various different types of correlations computed in RT research (e.g., correlations between a mean RT and an external non-RT measure, between two mean RTs, between a mean RT and an RT effect size). The analysis revealed that observed RT-based correlations depend on many parameters of the underlying processes contributing to RT. We conclude that these correlations often fail to support the inferences drawn from them and that their proper interpretation is far more complex than is generally acknowledged.

show abstract

Section: Reliability Of Mean Reaction Timessupporting

confidence: 68%

Mental chronometry and individual differences: Modeling reliabilities and correlations of reaction time means and effect sizes

2013

View full text Add to dashboard Cite

show abstract

“…Internal consistency reliability estimates, such as coefficient alpha, are based on the assumption that all items measure the same latent trait on the same scale, i.e., that the same latent trait equally contributes to all item scores (Yang and Green 2011). The multidimensional nature of SJTs violates this strict assumption resulting in an inaccurate estimate of reliability (Graham 2006). However, the integrity-based SJT used in this study was designed to measure one dimension, which might lead to a less serious violation of the assumption of unidimensionality.…”

Section: Present Studymentioning

confidence: 99%

Scoring method of a Situational Judgment Test: influence on internal consistency reliability, adverse impact and correlation with personality?

Leng

Stegers‐Jager

Husbands

et al. 2016

Adv in Health Sci Educ

View full text Add to dashboard Cite

Situational Judgment Tests (SJTs) are increasingly used for medical school selection. Scoring an SJT is more complicated than scoring a knowledge test, because there are no objectively correct answers. The scoring method of an SJT may influence the construct and concurrent validity and the adverse impact with respect to non-traditional students. Previous research has compared only a small number of scoring methods and has not studied the effect of scoring method on internal consistency reliability. This study compared 28 different scoring methods for a rating SJT on internal consistency reliability, adverse impact and correlation with personality. The scoring methods varied on four aspects: the way of controlling for systematic error, and the type of reference group, distance and central tendency statistic. All scoring methods were applied to a previously validated integrity-based SJT, administered to 931 medical school applicants. Internal consistency reliability varied between .33 and .73, which is likely explained by the dependence of coefficient alpha on the total score variance. All scoring methods led to significantly higher scores for the ethnic majority than for the non-Western minorities, with effect sizes ranging from 0.48 to 0.66. Eighteen scoring methods showed a significant small positive correlation with agreeableness. Four scoring methods showed a significant small positive correlation with conscientiousness. The way of controlling for systematic error was the most influential scoring method aspect. These results suggest that the increased use of SJTs for selection into medical school must be accompanied by a thorough examination of the scoring method to be used.

show abstract

“…This result is an underestimation of alpha as well as the reliability of the instrument (Green & Thompson, 2005). If the number of test items, on the other hand, is too small, the assumption of tau-equivalence will also be violated and this will result in an underestimation of reliability (Graham, 2006).…”

Section: Describes the Likert Scale Asmentioning

confidence: 99%

The Use Of Cronbach Alpha Reliability Estimate In Research Among Students In Public Universities In Ghana.

Quansah

2017

AJOTE

View full text Add to dashboard Cite

This study examines the use of Cronbach alpha reliability estimate in research among univers ity students in Ghana. An exploratory research design was employed in the study. University students' (both undergraduate and post-graduate) research works were selected from three public universities in Ghana. With the use of an inclusion criteria, purposive sampling technique was used to sample 100 research works conducted by students. The sampled research works were examined to evaluate how the students used Cronbach alpha reliability estimate. It was revealed that 91% of the works properly satisfied the conditions for the use of Cronbach alpha. However, 81% of the students calculated for the alpha for multiple constructs which suggests that alpha was treated as a measure of multidimensionality instead of internal consistency. It is recommended that applied courses in research and statistics should be mounted in the various public universities in Ghana. Workshops and seminars should also be organised for both students and lecturers on the use of Cronbach alpha reliability estimate.

show abstract

Congeneric and (Essentially) Tau-Equivalent Estimates of Score Reliability

Cited by 585 publications

References 15 publications

Mental chronometry and individual differences: Modeling reliabilities and correlations of reaction time means and effect sizes

Mental chronometry and individual differences: Modeling reliabilities and correlations of reaction time means and effect sizes

Scoring method of a Situational Judgment Test: influence on internal consistency reliability, adverse impact and correlation with personality?

The Use Of Cronbach Alpha Reliability Estimate In Research Among Students In Public Universities In Ghana.

Contact Info

Product

Resources

About