A perspective on situational judgment tests: From measures of situational judgment to measures of general domain knowledge

Shaw, Amy

doi:10.1016/j.paid.2021.110850

Cited by 4 publications

(19 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Patterson et al found that both SJT and MMI scores had strong correlations with the objective structured clinical examination (OSCE) scores (r = 0.44 and r = 0.46, respectively; P < 0.001 for both) during an end-of-training assessment. The researchers attributed these findings to the similarities between these methods because both extend beyond applied knowledge (23,24). For the same reasons, Patterson et al and Tiffin et al found SJT scores to be a strong predictor of practical assessment of clinical examination skills scores during an end-oftraining assessment, especially with construct-driven SJT (23,25).…”

Section: Discussionmentioning

confidence: 97%

Performance Prediction of the Family Medicine Applicants of Oman Medical Specialty Board in Multiple Mini-interviews Using the International Foundations of Medicine Exam and Situational Judgment Test

Hadhrami

Jaafar²,

Farsi³

et al. 2022

J Med Edu

View full text Add to dashboard Cite

Background: Entry into postgraduate medical training programs should be based on valid selection measures predictive of training performance. A situational judgment test (SJT) is often used during the residency selection process to measure knowledge and behavioral attributes. Objectives: This study aimed to test the validity of the International Foundations of Medicine (IFOM) examination and an SJT for predicting performance during multiple mini-interviews (MMI) of the applicants for a family medicine residency program in Oman. Methods: This retrospective cross-sectional study was conducted at Oman Medical Specialty Board (OMSB) during the application process for the academic year 2021. Data were collected from 80 applicants to the OMSB Family Medicine Residency Program. Before participating in the MMI panels, all applicants underwent pre-interview tests (comprising the IFOM examination and SJT). Multiple regression analysis was performed to determine the predictors of the MMI score. Results: A total of 67 applicants completed the entire selection process cycle and were included in the study. The SJT score was a significant predictor of the MMI score (r = 0.367; P = 0.002), whereas the IFOM score was not a significant predictor (r = 0.105; P = 0.396). Conclusions: This study supports using SJT as an additional selection measure alongside the MMI panel for entry to OMSB residency training.

show abstract

Section: Discussionmentioning

confidence: 97%

Performance Prediction of the Family Medicine Applicants of Oman Medical Specialty Board in Multiple Mini-interviews Using the International Foundations of Medicine Exam and Situational Judgment Test

Hadhrami

Jaafar²,

Farsi³

et al. 2022

J Med Edu

View full text Add to dashboard Cite

show abstract

“…However, if the true effectiveness of behaviors matters, ITP theory may be more closely specified to explain why individuals differ in their general domain knowledge about effective behaviors across situations. Recently, Shaw (2021) proposed that an integration of ITP theory into intellectual development theories may be one way to do so.…”

Section: Discussionmentioning

confidence: 99%

On the Construct-Related Validity of Implicit Trait Policies

2021

View full text Add to dashboard Cite

In response to recent calls to incorporate Implicit Trait Policies (ITPs) into personality research, the current study examined the construct-related validity of ITP measures. ITPs are defined as implicit beliefs about the effectiveness of behaviors that reflect a certain trait. They are assessed by utilizing the methodology of Situational Judgment Tests. We empirically examined ( N = 339) several underlying key assumptions of ITP theory, including trait-specificity, the relation to personality traits, their context-independence, and the relation to general domain knowledge. Overall, our results showed little support for these assumptions. Although we found some confirmation for expected correlations between ITPs and personality traits, most of the observed variance in ITP measures was either method-specific or due to measurement error. We conclude that the herein examined ITP measures lack construct-related validity and discuss implications for ITP theory and assessment.

show abstract

“…General domain knowledge represents common beliefs of behavior that is presumed effective across various situations ( Naemi et al, 2016 ; Torres and Beier, 2016 ). A very specific form of GDK is called an implicit trait policy or ITP ( Motowidlo et al, 2006 ; Shaw, 2021 ). An ITP represents self-beliefs about a causal relationship between personality traits and behavioral effectiveness ( Oostrom et al, 2012 ).…”

Section: Introductionmentioning

confidence: 99%

“… The implicit trait policy framework of intercultural effectiveness. This figure integrates the theoretical accounts of Leung et al (2014) (components and relations annotated with a 1) and Shaw (2021) (components and relations annotated with a 2) into the ITP framework of intercultural effectiveness, with an added feedback component as suggested by SCCT ( Lent et al, 1994 ), components and relations annotated with a 3. A high disposition on intercultural traits (e.g., cultural empathy, see van Oudenhoven and van der Zee, 2002 ) predicts both an ethnorelative attitude ( Bizumic and Duckitt, 2012 ) as well as more intercultural capabilities (e.g., cultural intelligence, see Schwarzenthal et al, 2019 ).…”

Section: Introductionmentioning

confidence: 99%

“…The first account is the framework of intercultural competence by Leung et al (2014) . The second account is the model by Shaw (2021) on how GDK as an ITP can lead to more effective behavior. As both accounts already summarize a vast amount of literature on intercultural competence and ITPs respectively, our intention is not to question these accounts and replicate the results of all hypotheses made.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Presenting the direct intercultural effectiveness simulation: an implicit trait policy on intercultural competence

Schelfhout

Derous

2023

Front. Psychol.

View full text Add to dashboard Cite

BackgroundAn implicit trait policy (ITP) represents the interaction between a personal disposition and general domain knowledge on how to effectively handle a specific (intercultural) situation. Such an ITP is a proven construct to create instruments that can predict future effective behavior. Moreover, such a simulation can provide valuable proxies for actual (future) behavior, as measures of (future) real life intercultural interactions are not always available.MethodsIn a series of three studies (N1 = 224, N2 = 291, N3 = 478), the present research introduces a “Direct Intercultural Effectiveness Simulation” or DIES, an instrument that simulates intercultural effectiveness by directly tapping into an ITP on intercultural competence.ResultsFirst and foremost, the present research demonstrates that the DIES instrument generates reliable and construct-valid measures of intercultural effectiveness. Second, the DIES instrument also shows expected converging and diverging patterns when correlated with a nomological network on intercultural effectiveness. And third, the DIES measure is further validated by integration into an ITP framework of intercultural effectiveness based on theoretical and empirical accounts from literature.ConclusionThe DIES instrument generates a reliable and valid measure of intercultural effectiveness by tapping into an ITP on intercultural competence. Theoretically, the present research integrates the instrument into literature by empirically verifying an ITP framework of intercultural effectiveness. In practice, the DIES instrument can be used as an awareness or training proxy for actual behavior to tackle important problems like ethnic prejudice and discrimination.

show abstract

A perspective on situational judgment tests: From measures of situational judgment to measures of general domain knowledge

Cited by 4 publications

References 10 publications

Performance Prediction of the Family Medicine Applicants of Oman Medical Specialty Board in Multiple Mini-interviews Using the International Foundations of Medicine Exam and Situational Judgment Test

Performance Prediction of the Family Medicine Applicants of Oman Medical Specialty Board in Multiple Mini-interviews Using the International Foundations of Medicine Exam and Situational Judgment Test

On the Construct-Related Validity of Implicit Trait Policies

Presenting the direct intercultural effectiveness simulation: an implicit trait policy on intercultural competence

Contact Info

Product

Resources

About