Examining Designed Experiences: A Walkthrough for Understanding Video Games as Performance Assessments

McCreery, Michael P.; Schrader, P. G.; Krach, S. Kathleen; Laferriere, Jeffrey R.; Bacos, Catherine A.; Fiorentini, Joseph P.

doi:10.1007/978-3-030-15569-8_6

Cited by 7 publications

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, we chose to apply validation and reliability methods to COTS games, given the widespread use of COTS games for assessment in research (McCreery et al, 2019;Schrader et al, 2019;Shute et al, 2013). Commercial organizations are reducing the drawbacks of using GBAs by developing more cost-effective and accessible GBAs for organizational use (e.g., Arctic Shores, Revelian, Pymetrics).…”

Section: Limitationsmentioning

confidence: 99%

Virtual reality assessments (VRAs): Exploring the reliability and validity of evaluations in VR

Sanchez

Weiner²,

Zelderen

2022

Int J Selection Assessment

View full text Add to dashboard Cite

Virtual reality (VR) is a potential assessment format for constructs dependent on certain perceptual characteristics (e.g., realistic environment and immersive experience). The purpose of this series of studies was to explore methods of evaluating reliability and validity evidence for virtual reality assessments (VRAs) when compared with traditional assessments. We intended to provide the basis of a framework on how to evaluate VR assessments given that there are important fundamental differences to VR assessments compared with traditional assessment formats. Two commercial off-the-shelf (COTS) games (i.e., Project M and Richie's Plank Experience)were used in Studies 1 and 2, while a game-based assessment (GBA; Balloon Pop, designed for assessment) was used in Study 3. Studies 1 and 2 provided limited evidence for the reliability and validity of the VRAs. However, no meaningful constructs were measured by the VRA in Study 3. Findings demonstrate limited evidence for these VRAs as viable assessment options through the validity and reliability methods utilized in the present studies, which in turn emphasize the importance of aligning the assessment purpose to the unique advantages of a VR environment. K E Y W O R D Scommercial-off-the-shelf (COTS) games, game-based assessment (GBA), reliability, validity, virtual reality (VR) Practitioner points• Findings were mixed in correlating the VRA scores with similar assessments to the intended constructs being measured.• Details are provided on the design and scoring for the presented VRAs.• Although research using VRAs is still preliminary, there are promising methods through which we might design unique behavior based evaluation. | INTRODUCTIONThe measurement of trait differences using prehire assessments is widely practiced and useful to help differentiate job candidates based on constructs that are related to job performance, such as personality traits like conscientiousness (r = .31) and aptitudes like cognitive ability (r = .51;Hough et al., 2001). The self-report method has demonstrated accurate measurement for certain constructs in various contexts (Chan, 2009

show abstract

Section: Limitationsmentioning

confidence: 99%