Generalizability theory.

Abstract

In this paper we review generalizability (G) theory, a theory of the multifaceted errors of a behavioral measurement. The review was undertaken at the request of Philip Levy, then editor of the Journal.His idea was that the review would commemorate the first article on G theory, which the Journal published in 1963 (Cronbach, Rajaratnam, & Gleser, 1963). For these and personal reasons, we undertook the review. The review does not cover the period 1963-1972 because that has already been done by Cronbach, Gleser, Nanda, and Rajaratnam (1972).In Section 1 we sketch out generalizability theory for those who are not familiar with it. In doing so, we summarize the notation used in the review. Section 2 reviews theoretical contributions. While it primarily reflects what has been published, we take up some new topics and identify others in need of treatment. Section 3 presents an application of the theory in some detail. This application illustrates basic concepts in the theory (Section 1) as well as recent theoretical contributions (Section 2

Everything that you have ever been told about assessment center ratings is confounded.

“…In the discussion above, we refer to generalization to different assessors only. 6 We define reliability here in relative, rather than in absolute, terms (see Shavelson & Webb, 1991). required us to average AC ratings (or post exercise dimension ratings, PEDRs) across exercises.…”

“…To assuage concerns about the number of dimensions and exercises in our study relative to the other examples listed in Table 5, we applied a decision study from generalizability theory methodology, which is akin to applying the Spearman-Brown prediction formula for multifaceted measures (Brennan, 2001;Shavelson & Webb, 1991). Using a decision study, we extrapolated from our observed reliability outcomes based on three exercises and six dimensions to an alternative design based on five exercises and 13 dimensions: the largest number of exercises and dimensions used in the studies listed in Table 5 (i.e., in the Bowler & Woehr, 2009, study).…”

Application of a new method in the study of pelvic floor muscle passive properties in continent women

“…We also reported median scores since some parameters were not normally distributed. In condition 3 (lengthening and shortening cycles), the systematic difference across cycles was assessed using a type II intra-class coefficient (Shrout and Fleiss, 1979) and standard error of measurement (SEM) (Shavelson, 1991). The purpose of these analyses was to assess which cycles should be averaged in order to have more stable and representative passive parameters of the PFMs.…”

The association of parental characteristics and psychological problems in obese youngsters

“…Inter-rater and test-retest reliability of the subscale scores on the EDE were calculated, whereby generalizability theory was used as a framework for the reliability estimation. 27 The inter-rater generalizability was quite high: Restraint: G ¼ 0.99; Eating concern: G ¼ 0.97; Weight concern: G ¼ 0.97; Shape concern: G ¼ 0.91 and objective bulimic episodes: G ¼ 0.98. The testretest generalizability was very good: Restraint: G ¼ 0.78; Eating concern: G ¼ 0.77; Weight concern: G ¼ 0.61; Shape concern: G ¼ 0.83 and objective bulimic episodes: G ¼ 0.68.…”