Beyond the Spearman–Brown: A structural approach to maximal reliability.

Abstract

The requirement of parallel parts has long been the cornerstone of classic reliability theory. By recasting reliability in a structural equation framework, items, raters, or judges no longer need to be treated as equivalent entities. Instead, unique reliability estimates can be determined for each and collectively used to assess the maximal reliability of a weighted composite, with the composite reliability submitted to inferential test. Procedures are shown to generalize from single to multifactor applications. Ramifications of a structural approach to reliability determination are probed, and the dilemma posed by possible falsification of the true score hypothesis presented for individual researcher consideration.

A dimensional analysis of creativity and mental illness: Do anxiety and depression symptoms predict creative cognition, creative accomplishments, and creative self-concepts?

“…The faceted design-four raters crossed by three tasks-makes traditional reliability statistics (e.g., Cronbach's alpha) inappropriate (Drewes, 2000). Moreover, there is controversy over whether divergent thinking tasks are interchangeable-they seem to represent fixed rather than random effects (Almeida, Prieto, Ferrando, Oliveira, & Ferrándiz, 2008;Silvia et al, 2008, Study 1).…”

“…Moreover, there is controversy over whether divergent thinking tasks are interchangeable-they seem to represent fixed rather than random effects (Almeida, Prieto, Ferrando, Oliveira, & Ferrándiz, 2008;Silvia et al, 2008, Study 1). For the present design, an analogous and appropriate reliability statistic is maximal reliability (Drewes, 2000), known as H, which represents -the degree to which the indicators can capture information about the underlying factor‖ (Gagné & Hancock, 2006, p. 68). Like Cronbach's alpha, H expresses reliability in a 0 to 1 range, with higher values indicating higher reliability.…”

Development of Experimental Army Enlisted Personnel Selection and Classification Tests and Job Performance Criteria

“…Discard non-traditional items with low loadings. Estimate reliabilities of the retained non-traditional items by squaring their loadings on the latent factor (Drewes, 2000).…”

A snapshot of creativity: Evaluating a quick and simple method for assessing divergent thinking

“…Overall, this method appears promising. First, the factor structure of the snapshot scores showed good construct reliability (Drewes, 2000)-all of the H values were over .80, so good score reliability was achieved with only three raters and two tasks. Second, good evidence for concurrent validity was found.…”

“…Nevertheless, an analogous statistic for latent variable models is maximal reliability (Drewes, 2000), which represents "the degree to which the indicators can capture information about the underlying factor" (Gagné & Hancock, 2006, p. 68). The higher order creativity variable (H = .83) and the lower order brick (H = .84) and knife (H = .88) variables displayed good reliability.…”