Video‐based Situational Testing

Abstract

In two separate studies, video‐based tests of situational judgment were developed and validated against measures of performance for hourly service workers. In the first study, 684 employees were used to develop a test of retail associate judgment and 787 newly hired employees were used to cross‐validate the instrument. In the second study, 412 current employees were used to develop a test of nursing home caregiver judgment and 148 newly hired caregivers were used to cross‐validate this video‐based test. In both studies, responses to video‐based situational vignettes were empirically keyed against supervisory ratings of performance. The resulting keys produced uncorrected cross‐validities in the low .20s. The video‐based test scores were also found to be related to measures of cognitive ability and, to a lesser extent, experience. These results suggest that cognitive ability and possibly experience account for some but not all of the predictiveness of video‐based situational tests. Video‐based situational tests demonstrated score differences between Whites and non‐Whites of roughly one‐half a standard deviation, indicating that the use of such tests could produce adverse impact against non‐Whites. In the second study, customer preferences regarding desired behavior were also used to develop a rational scoring key. This customer driven key was significantly related to performance (r= .33) in the cross‐validation sample, demonstrating that customers' preferences can provide the basis for developing valid predictors. Potential benefits of video‐based situational tests in selection and directions for future research are discussed.

Situational judgment tests: a review of recent research

“…In the meta-analysis of , it was found that SJTs show a correlation of 0.46 with cognitive ability, even though there was substantial variability around this estimate. For instance, video-based SJTs had lower correlations with cognitive ability than written SJTs (Weekley and Jones, 1997). Another example is that SJTs based on a job analysis were usually more highly related to cognitive ability than those not based on a job analysis (0.50 versus 0.38).…”

“…Each scenario describes a person handling a typical job-related situation. At a critical "moment of truth", the scenario ends and the applicant is asked to choose among several courses of action (Dalessio, 1994;Smiderle et al, 1994;Weekley and Jones, 1997). Questions and response options are presented visually and supported by narration.…”

Situational Tests in Student Selection: An Examination of Predictive Validity, Adverse Impact, and Construct Validity

“…Anastasi and Urbina define a situational test as`one that places the test taker in a situation closely resembling or simulating a``real-life'' criterion situation ' (1997, p. 450). Given this definition, examples of situational tests include accomplishment records (e.g., Hough 1984), situational judgement tests (e.g., Motowidlo, Dunnette and Carter 1990), situational interviews (e.g., Latham, Saari, Pursell and Champion 1980), video-based situational judgement tests (e.g., Weekley and Jones 1997), work samples (e.g., Robertson and Kandola 1982), and assessment centre exercises such as role-plays (e.g., Thornton 1992). Situational tests have also become known in the educational literature under the aliases of performance assessment, alternative assessment or authentic assessment (Baker, O'Neil and Linn 1993;Linn, Baker and Dunbar 1991;Messick 1994;Sackett 1998;Wiggins 1989).…”

Fundamentos e aplicabilidade dos Testes de Julgamento Situacional no contexto da Psicologia Organizacional

“…O que parece ser uma concordância na literatura é que os TJS são importantes devido a sua capacidade pre ditora do comportamento humano para a situação exposta (Hanson & Ramos, 1996;McDaniel, Morgeson, Finnegan, Campion, & Braverman, 2001;Phillips, 1992Phillips, , 1993Weekley & Jones, 1997, 1999. Esse compor tamento, entretanto, pode ou não estar relacionado a processos de seleção, ao ambiente de trabalho e, além disso, pode ou não ser aquele que seria assumido pelo indivíduo avaliado, conforme já visto no que tange às instruções de respostas.…”