Communicating the validity of structured job interviews with graphical visual aids

Abstract

The structured job interview is often resisted by human resource (HR) professionals despite its superior predictive validity compared to the traditional unstructured interview. However, HR professionals may underestimate the utility of structured interviews because of how validity information is presented. Three online experiments with MTurk samples were conducted to examine the effect of statistical visual aids (Binomial effect size display; expectancy chart; icon array) on people's perceived usefulness of validity information and intentions toward using the structured interview. Results showed that graphical visual aids (expectancy chart and icon array) were perceived as more useful for communicating validity evidence than a nongraphical effect size display. People also judged the structured interview as more useful and were more willing to use it when validity evidence was presented graphically. Individual differences in graph literacy and nationality were also examined.

Implementing evidence-based assessment and selection in organizations: A review and an agenda for future research

“…Overall, nine studies showed that the effects of different presentation modes of validity information were small and seemed to depend on individual and demographic differences. Furthermore, a complication of using tabular or visual displays to communicate validity evidence in the form of the expected number of successful hires is that a base rate needs to be assumed, which can substantially differ across job selection contexts (50% was assumed in Zhang et al, 2018). The results also suggest that displaying a range of validity coefficients of possible assessment instruments facilitates decision makers' evaluation of validity information.…”

Telling stories to communicate the value of the pre-employment structured job interview

“…Nevertheless, research consistently shows that increasing the use of structured interview could significantly enhance hiring outcomes (Huffcutt & Arthur, 1994;Lubbe & Nitsche, 2019) and reduce adverse impact (Huffcutt & Roth, 1998;Williamson, Campion, & Malos, 1997). Indeed, a bad hire can incur a significant financial cost (Forbes, 2016) have a negative impact on workplace morale and productivity (Dunlop & Lee, 2004) Efforts to combat such resistance have primarily focused on more effective means of presenting statistical evidence (Highhouse et al, 2017;Zhang et al, 2018) or using alternative numerical metrics such as utility analysis (Macan & Highhouse, 1994) to better communicate the value of evidence-based practices to relevant stakeholders. Zhang and colleagues (2018), for example, found that managers were more willing to adopt structured interviews when validity information was presented using alternative graphical displays.…”

“…To date, prescriptions for overcoming managers' resistance toward evidence-based hiring practices have primarily focused on more efficient ways to communicate statistical validity evidence (e.g., Highhouse, Brooks, Nesnidol, & Sim, 2017;Zhang, Highhouse, Brooks, & Zhang, 2018). Statistical evidence, however, is not the only tool for persuasion.…”

Telling stories to communicate the value of the pre‐employment structured job interview

“…To date, prescriptions for overcoming managers’ resistance toward evidence‐based hiring practices have primarily focused on more efficient ways to communicate statistical validity evidence (e.g., Highhouse, Brooks, Nesnidol, & Sim, ; Zhang, Highhouse, Brooks, & Zhang, ). Statistical evidence, however, is not the only tool for persuasion.…”

“…Efforts to combat such resistance have primarily focused on more effective means of presenting statistical evidence (Highhouse et al, ; Zhang et al, ) or using alternative numerical metrics such as utility analysis (Macan & Highhouse, ) to better communicate the value of evidence‐based practices to relevant stakeholders. Zhang and colleagues (), for example, found that managers were more willing to adopt structured interviews when validity information was presented using alternative graphical displays. The persuasive capacity of statistical evidence, however, may be limited: Many managers are not sufficiently trained in statistics or fluent in numerical comprehension (Best, ; Kirsch, Jungeblut, Jenkins, & Kolstad, ).…”