A systematic literature review on serious games evaluation: An application to software project management

Calderón, Alejandro; Ruiz, Mercedes

doi:10.1016/j.compedu.2015.07.011

Cited by 213 publications

(182 citation statements)

References 76 publications

Supporting

Mentioning

149

Contrasting

Unclassified

Order By: Relevance

“…The methods used to measure their effectiveness vary from study to study (some standard guidelines are starting to arise [6]). However, a large number of serious games research studies primarily depend on data from surveys and questionnaires [7].…”

Section: Introductionmentioning

confidence: 99%

“…Current research shows that questionnaires, which are outside the game context, are the most common method to collect data in formal experiments with serious games [7]. This heavily contrasts with the practices in non-formal environments.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper we build upon a previous review by Calderon and Ruiz on serious games and evaluation [7]. They identified a total of 120 papers using key search terms such as "evaluation / validation / assessment" combined with "serious games / simulation games".…”

Section: Interactions Tracked In Serious Gamesmentioning

confidence: 99%

See 2 more Smart Citations

Applying standards to systematize learning analytics in serious games

Serrano-Laguna

Martínez‐Ortiz

Haag

et al. 2017

Computer Standards & Interfaces

110

View full text Add to dashboard Cite

Learning Analytics is an emerging field focused on analyzing learners’ interactions with educational content. One of the key open issues in learning analytics is the standardization of the data collected. This is a particularly challenging issue in serious games, which generate a diverse range of data. This paper reviews the current state of learning analytics, data standards and serious games, studying how serious games are tracking the interactions from their players and the metrics that can be distilled from them. Based on this review, we propose an interaction model that establishes a basis for applying Learning Analytics into serious games. This paper then analyzes the current standards and specifications used in the field. Finally, it presents an implementation of the model with one of the most promising specifications: Experience API (xAPI). The Experience API relies on Communities of Practice developing profiles that cover different use cases in specific domains. This paper presents the Serious Games xAPI Profile: a profile developed to align with the most common use cases in the serious games domain. The profile is applied to a case study (a demo game), which explores the technical practicalities of standardizing data acquisition in serious games. In summary, the paper presents a new interaction model to track serious games and their implementation with the xAPI specification.This study is part of the RAGE project. The RAGE project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 644187. This publication reflects only the author's view. The European Commission is not responsible for any use that may be made of the information it contains

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Applying standards to systematize learning analytics in serious games

Serrano-Laguna

Martínez‐Ortiz

Haag

et al. 2017

Computer Standards & Interfaces

110

View full text Add to dashboard Cite

show abstract

“…These games assist learners to understand a particular subject, help them in acquiring new skills, and expand their existing concepts (Dempsey et al, 1996). By playing the games, learners can experience a simulated real world problem (Calderón and Ruiz, 2015) without exposure to the costs and risks associated with it (Dantas et al, 2004). Serious games can be played with multiple players, such as the Virtual Construction Negotiation (VCON) game (Yaoyuenyong et al, 2005), or they can only be played by a single player, such as the Virtual Construction Simulator (VCS) game (Nikolic et al, 2010).…”

Section: Characteristics Of Serious Gamesmentioning

confidence: 99%

Design and evaluation of the project and program crashing games

Rumeser¹,

Emsley

2018

JARHE

View full text Add to dashboard Cite

Purpose -This research aims to help project management game designers and educators in simulating complexity in project management games and in assessing the effect of simulated project complexity levels on students' learning experience. To achieve this aim, we attempt to design and evaluate two computer-based project crashing games with different complexity levels, namely Project Crashing Game (PCG) and Program Crashing Game (PgCG).Design/methodology/approach -A literature review is conducted to identify serious games design principles. These principles are then manifested in the design of PCG and PgCG. The latter is a more complex version of the first. Students' reaction after playing both games are then analyzed quantitatively.Findings -We discover that students' learning experience is affected by how complex the simulated project is. The more complex the project is (i.e. as in the PgCG), the more realistic the game is perceived. Nevertheless, we also discover that the less complex game (PCG) offers significant value to students, particularly to teach basic project management principles to those with minimum or no practical experience. This game is perceived as better in increasing students' learning confidence as its content is perceived as more relevant to their existing knowledge.Originality/value -We adopt a project complexity perspective when designing and evaluating the games.

show abstract

“…Regarding the scope of serious games for training SPM, there are a limited number of games available (Caulfield, Xia, Veal, & Maj, 2011;Calderón & Ruiz, 2015;Petri & von Wangenheim, 2017), with well-known examples being: SimSoft , SimSE (Navarro & Hoek, 2004), X-MED (Von Wangenheim, Thiry, & Kochanski, 2009), DELIVER! (Von Wangenheim, Savi, & Borgatto, 2012).…”

Section: Introductionmentioning

confidence: 99%

ProDecAdmin: A Game Scenario Design Tool for Software Project Management Training

Calderón

Ruiz

O’Connor

2017

Communications in Computer and Information Science

Self Cite

View full text Add to dashboard Cite

Abstract. Teaching Software Project Management (SPM) for InformationTechnology (IT) learners is a relevance issue. The necessity of teaching SPM in a highly practical way moves trainers towards the use of new methods and techniques such as simulations, serious games or gamification strategies. The majority of the existing serious games for SPM do not offer flexibility, in terms of the ability to provide and dynamically change game scenarios during the life of the serious game, and not allow assessing learners' new skill automatically. In this paper, we introduce the administration tool of the serious game ProDec, that allow trainers to design the game scenarios of the game trying to overcome the lacks found in the scope of serious games for SPM.

show abstract

A systematic literature review on serious games evaluation: An application to software project management

Cited by 213 publications

References 76 publications

Applying standards to systematize learning analytics in serious games

Applying standards to systematize learning analytics in serious games

Design and evaluation of the project and program crashing games

ProDecAdmin: A Game Scenario Design Tool for Software Project Management Training

Contact Info

Product

Resources

About