Moving Forward Indirectly: Reanalyzing the validity of employment interviews with indirect range restriction methodology

Abstract

This study provides updated estimates of the criterion-related validity of employment interviews, incorporating indirect range restriction methodology. Using a final dataset of 92 coefficients (N = 7,389), we found corrected estimates by structural level of .20 (Level 1), .46 (Level 2), .71 (Level 3), and .70 (Level 4). The latter values are noticeably higher than in previous interview meta-analyses where the assumption was made that all restriction was direct. These results highlight the importance of considering indirect range restriction in selection. However, we found a number of studies involving both indirect and direct restriction, which calls into question the viability of assuming all restriction is now indirect. We found preliminary empirical support for correction of one of these multiple restriction patterns, indirect then direct.

A meta‐analysis of interview length on reliability and validity

“…This difference is likely due to the increased number of highly structured interviews in their sample. Forty-six of the 73 studies (63%) in the Huffcutt et al (2014) sample were at the highest level of structure, whereas only 27 of 64 studies (42%) in the current study were highly structured.…”

“…Using inter-rater reliability estimates from Table 1, corrections were made at the different levels of structure (low = .50, medium = .61, and high = .84). A range restriction ratio of .74 was used following Huffcutt et al (2014). Finally, the value of .52 was used to correct for unreliability in the criterion (Huffcutt et al, 2014;Rothstein, 1990).…”

“…The relatively modest effect of structure on validity may reflect that a certain amount of structure is useful but past a certain point does not lead to higher validity (Huffcutt & Arthur, 1994;Huffcutt et al, 2014). Interview structure may improve validity primarily through the increased reliability of structured interviews (Schmidt & Zimmerman, 2004).…”

“…Not only do employers need to recruit qualified applicants, but they need to identify the best applicants from a potentially large pool of qualified candidates. The employment interview, especially when structured, has been shown to be a good predictor of job performance (Huffcutt & Arthur, 1994;Huffcutt, Culbertson, & Weyhrauch, 2014;Marchese & Muchinsky, 1993;McDaniel, Whetzel, Schmidt, & Maurer, 1994), but can be quite costly to develop and implement. The development of an interview, especially a structured interview, requires careful development of interview questions and scoring keys for responses.…”

17. Functional forms of competence: Interviewing

“…In terms of how structured interviews fare in practice, researchers have consistently found that highly structured interviews, compared to their unstructured counterparts, are preferable as they tend to be more reliable (Conway, Jako, and Goodman 1995;Huffcutt, Culbertson, and Weyhrauch 2013), better predictors of job performance (Huffcutt, Culbertson, and Weyhrauch 2014;Wiesner and Cronshaw 1988), and fairer, with fewer mean differences on the basis of sex and race (Huffcutt and Roth 1998).…”