Putting judging situations into situational judgment tests: Evidence from intercultural multimedia SJTs.

Abstract

Although the term situational judgment test (SJT) implies judging situations, existing SJTs focus more on judging the effectiveness of different response options (i.e., response judgment) and less on how people perceive and interpret situations (i.e., situational judgment). We expand the traditional SJT paradigm and propose that adding explicit assessments of situational judgment to SJTs will provide incremental information beyond that provided by response judgment. We test this hypothesis across 4 studies using intercultural multimedia SJTs. Study 1 uses verbal protocol analysis to discover the situational judgments people make when responding to SJT items. Study 2 shows situational judgment predicts time-lagged, peer-rated task performance and interpersonal citizenship among undergraduate seniors over and above response judgment and other established predictors. Study 3 shows providing situational judgment did not affect the predictive validity of response judgment. Study 4 replicates Study 2 in a working adult sample. We discuss implications for SJT theory as well as the practical implications of putting judging situations back into SJTs.

In Defense of the Situation: An Interactionist Explanation for Performance on Situational Judgment Tests

“…Consistency in SJT performance for versions with and without prompts (Krumm et al, 2015) might reflect the previously noted issue that situational cues are still present and clear in the response options of SJTs, such that the two versions do not reflect a substantial difference in situational strength (i.e., general domain knowledge is important in both measures). This interpretation of the role of situational strength is consistent with Rockstuhl et al's (2015) finding that SJTs that explicitly ask respondents to judge the situation provide incremental validity over SJTs that do not, as well as consistent with studies showing the importance of situational cues in other assessment formats (Jansen et al, 2013;Melchers et al, 2012). Further, we suggest that all SJTs invoke situational cues that are stronger than in self-report personality tests, which may account for the incremental validity of SJTs above and beyond personality tests (Meriac, Hoffman, Woehr, & Fleisher, 2008).…”

Why Situational Judgment Is a Missing Component in the Theory of SJTs

“…Thus, similar to the available evidence that situation assessment correlates with interview or AC performance (cf. Kleinmann et al, 2011), we would consider the findings by Rockstuhl et al (2015) as support for the suggestion that SJT scores that are based on effectiveness judgments nevertheless capture systematic variance that is related to situational judgment.…”

Supporting Higher Education—Hospital Transition through Blended Learning

“…For each situation described, there are multiple and alternative actions. The test taker has to make judgments about the course of actions presented and choose the most appropriate response using a forced-choice or Likert-scale format (Rockstuhl et al, 2015). Nine experts were asked to assess the content validity of the tool by rating scenarios and alternative courses of actions (Lynn, 1986).…”