A meta‐analysis of interview length on reliability and validity

Abstract

First impressions are frequently seen as a biasing factor that may prevent an interviewer from forming a comprehensive assessment of the applicant. However, research has found that people can make surprisingly accurate impressions of others based on minimal information. Additional exposure to the applicant would be expected to lead to a more accurate impression, but a previous meta-analysis on the employment interview found evidence for a negative relationship between the length of the interview and validity. Using a much larger sample of studies, the current meta-analysis examined whether length of the interview was related to the validity of the interview. In addition, I examined the relation between interview length and reliability. Results revealed that the subset of studies reporting length of interviews yielded reliability and validity estimates consistent with more comprehensive meta-analyses. Consistent with previous research on consensus judgements of personality, length of the interview was found to be unrelated to reliability. However, contrary to the previous meta-analysis, validity was also unrelated to the length of the interview. Practitioner pointsLittle attention has been given to how the length of time spent interviewing job candidates relates to the reliability and validity of interviews Length of the interview was not related to reliability or validity of interviews Results suggest that organizations can implement shorter interviews to save time and money compared to longer interviews, without sacrificing reliability and predictive validity Selecting a new employee can be a time-consuming process. Not only do employers need to recruit qualified applicants, but they need to identify the best applicants from a potentially large pool of qualified candidates. The employment interview, especially when structured, has been shown to be a good predictor of job performance (Huffcutt & Arthur, 1994;Huffcutt, Culbertson, & Weyhrauch, 2014;Marchese & Muchinsky, 1993;McDaniel, Whetzel, Schmidt, & Maurer, 1994), but can be quite costly to develop and implement. The development of an interview, especially a structured interview, requires careful development of interview questions and scoring keys for responses. Interviewers, who are often highly paid managers and executives, may spend considerable time in training to administer and score the interview as well as time spent administering the interview to numerous applicants. Even an unstructured interview will incur considerable costs in terms of time spent by interviewers in reviewing candidates' application materials prior to the interview and administering the interview. These increased costs may be acceptable if longer interviews result in improved psychometric properties (i.e., greater reliability and validity). The literature on the employment interview, however, provides scant attention to the relationship of interview length to the reliability and validity of the interview. Given the high costs of the employment interview, organizations could benefit...

Who will go the extra mile? Selecting organizational citizens with a personality-based structured job interview

“…Third and relatedly, the focal relationships between interview ratings of personality traits and supervisor ratings of OCBs were only moderate ranging from r = .22 to r = .30. These effect sizes can be categorized as small to medium (Bosco, Aguinis, Singh, Field & Pierce, 2015 ; Cohen, 1992 ) and are slightly lower than uncorrected estimates reported in the latest meta-analyses on the validity of medium to highly structured job interviews ranging from = .25 to = .36 (Huffcutt et al, 2014 ; Thorsteinson, 2018 ). One explanation may be the heterogeneity of the present sample (i.e., interviewees held different jobs and interview questions were not tailored to meet unique demands of these specific jobs).…”

Initial impression formation during the job interview: anchors that drive biased decision-making against stigmatized applicants

“…They conclude that "fast and frugal" heuristic judgements made during the rapport-building stage influence interviewer decision-making, raising concerns that bias might originate from initial impressions that are formed during rapport building (Levashina et al, 2014). In line with this, a recent meta-analysis shows that the length of the interview is unrelated to interview validity and reliability providing a different type of evidence that decisions may be made early in the interview process (Thorsteinson, 2018).…”

“…Moreover, investigating underlying cognitive and motivational processes of biased selection decisions against stigmatized vs. non-stigmatized applicants may lead to more generalizable and fundamental insights in hiring discrimination that could be applied to many different marginalized groups in many different contexts (Derous et al, 2013). Finally, literature on interview validity and bias typically consider characteristics of the applicant (e.g., Segrest Purkiss et al, 2006) and the tool (e.g., interview structure; Levashina et al, 2014;Thorsteinson, 2018) Although recently more attention goes to interviewer characteristics (Florea et al, 2019;Frieder et al, 2016), much to our surprise, one of the latest comprehensive reviews that incorporated interviewer characteristics has been published already some time ago (Posthuma et al, 2002). We suggest hiring discrimination literature to incorporate recruiter characteristics and decision-making theory in a more systematic way to increase our understanding of interview bias (Buijsrogge et al, 2016;Florea et al, 2019), and bias in other recruitment tools that are based on human decisionmaking (e.g., resume-screening; Derous & Ryan, 2019).…”

Predictive Validity of Interviewer Post-interview Notes on Candidates’ Job Outcomes: Evidence Using Text Data From a Leading Chinese IT Company

“…Although using the radar chart from job analysis to construct interview questions increases interview validity, how to use the chart more effectively needs further investigation. Since longer interviews cost the organization more ( Thorsteinson, 2018 ), interviewers are not likely to spend much time on an interview. Within a limited length of interview time, interviewers can either go broader to cover more aspects of the radar chart from job analysis and get a comprehensive impression of the candidate or select certain aspects to get a deeper understanding of the candidate’s capabilities.…”

“…However, only indirect effects have been found between job analysis and interview validity ( Conway et al, 1995 ). Extant research mostly uses cognitive test results as the dependent variable to show the validity of using job analysis in structured interviews ( Thorsteinson, 2018 ). These cognitive performance scores are then linked to the candidate’s job performance.…”