Applying Evidence-Centered Design to Measure Psychological Resilience: The Development and Preliminary Validation of a Novel Simulation-Based Assessment Methodology

Abstract

Modern technologies have enabled the development of dynamic game- and simulation-based assessments to measure psychological constructs. This has highlighted their potential for supplementing other assessment modalities, such as self-report. This study describes the development, design, and preliminary validation of a simulation-based assessment methodology to measure psychological resilience—an important construct for multiple life domains. The design was guided by theories of resilience, and principles of evidence-centered design and stealth assessment. The system analyzed log files from a simulated task to derive individual trajectories in response to stressors. Using slope analyses, these trajectories were indicative of four types of responses to stressors: thriving, recovery, surviving, and succumbing. Using Machine Learning, the trajectories were predictive of self-reported resilience (Connor-Davidson Resilience Scale) with high accuracy, supporting construct validity of the simulation-based assessment. These findings add to the growing evidence supporting the utility of gamified assessment of psychological constructs. Importantly, these findings address theoretical debates about the construct of resilience, adding to its theory, supporting the combination of the “trait” and “process” approaches to its operationalization.

Are two naïve and distributed heads better than one? Factors influencing the performance of teams in a challenging real-time task

“…In the present study, we employed a high-fidelity driving simulation that we developed in previous research (Kleitman et al, 2022) . Simulations are open-ended, rule-based discovery spaces that allow players to engage with artificial problems that result in quantifiable outcomes (Salen et al, 2004, p. 80).…”

“…The different operational conditions allowed us to examine whether the 2HBT1 effect and the relationship between communication and performance depended on the characteristics of a task which may introduce an informational advantage. The simulation is described in further detail in the Methods section (see also Kleitman et al, 2022).…”

“…We employed a driving simulation that we developed in previous research (Kleitman et al, 2022). During this simulation, participants completed a simulated emergency driving course in an urban environment as either an individual or an asymmetrical, distributed two-person team.…”