Game-based assessment: an integrated model for capturing evidence of learning in play

Abstract

The GBA design is grounded in recent research in game-based learning, evidence-centered design (ECD), and education data mining (EDM). We use a game-based learning experience to implement a version of ECD's task/content/evidence model into the game design. We then collect patterns of click-stream data, as in EDM, to develop records of in-game player interaction that can be used as evidence for learning. Here we briefly review some of the core research ideas that led to our GBA design. Video games and learning. Kurt Squire asserts that "games differ from simulations in that they give roles, goals, and agency", and use "transgressive play" (Salen & Zimmerman, 2004) to "elicit fantasies" (Squire, 2011, p. 29). A key aspect of effective game design is the integration of data channels that inform both play and design. Gee (2005) highlights how good video games include just-in-time information (scaffolding) and cycles of expertise. Good games include formative assessment cycles that foster ongoing feedback and customize player difficulty levels (Shute, 2011). In order to maintain this immersive context for learning, good games consist of ongoing assessment balanced with engaging mechanics and narrative (Squire, 2006). Good games are not only scaffolded, engaging designed experiences (Squire, 2006), they also hold the power to improve learning. Situated learning theory suggests that learning exists in situ, inseparable from environment or context (Brown & Collins, 1989). Virtual game worlds have been shown to provide a powerful environment for learning, supporting apprenticeship and collective higher-order thinking skills

The many faces of scientific inquiry: Effectively measuring what students do and not only what they say

“…Although research investigating the design and validity of STEM assessments in virtual environments with rich interactions, simulation and virtual laboratory experiences is still its infancy, several studies have attempted to measure learning and development using game and simulation data in a variety of ways. Some studies have used time and frequency variables as predictors for learning (Chin, Blair, & Schwartz, ; DiCerbo, ; Galaup et al, ; Lee et al, 2014; Liu et al, ; Shute & Wang, ), while other studies have examined the significance of sequence of choice and user path selections in virtual environments (Halverson & Owen, ; Loh & Sheng, ). In addition, some research has attempted to make connections between behaviors in the virtual environment and desired learning outcomes (Gobert et al, ; Quellmalz et al, ).…”

Exploring the influence of game design on learning and voluntary use in an online vascular anatomy study aid

“…For example, Halverson and Owen (2014) designed and tested a game, Progenitor X, with high school students to examine the link between gameplay performance and learning about stem cell biology. They developed their own framework ADAGE (Assessment Data Aggregator for Game Environments), based largely on evidence-centred design and educational data mining methodologies.…”

The Evidence Trace File: A Data Structure for Virtual Performance Assessments Informed by Data Analytics and Evidence‐Centered Design

“…Thus an appropriate set of attributes and parameters for each state needs to be established for each activity in collaboration between the members of the interdisciplinary design team for the VPAs. In early stages of development, finer grained, lower level events may need to be included in the ETF so patterns among them can be further explored to determine exactly which actions and system state variables are needed to produce the required evidence (e.g., DiCerbo et al, ; Halverson & Owen, ; Roberts, Chung, & Parks, ).The subsequent operational version of the ETF may then need to contain only higher level computed events, if the required computation is straightforward to calculate.…”