Employing game analytics techniques in the psychometric measurement of game-based assessments with dynamic content

Jaffal, Yasser; Wloka, Dieter W.

doi:10.20368/1971-8829/1063

Cited by 4 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To inform the design of valid assessments for immersive learning environments, a few frameworks and systems like the information trails (Loh, 2012) and the Evidence Centred Design (ECD) framework (Almond et al, 2003) have been used. Of these, the most commonly used embedded assessment design framework for immersive environments is the ECD framework (see Jaffal & Wloka, 2015;Kerr & Chung, 2012;Shute & Rahimi, 2021) The ECD framework is also the basis for the design of Stealth Assessment which is popular in the field of game-based learning and assessment (Alcañiz et al, 2018;Min et al, 2020;Smith et al, 2019). Although successfully used to design assessments for immersive environments, the design process is often complex and time-consuming and requires advanced statistical and machine-learning skills (Kim et al, 2016;Westera, 2019;Westera et al, 2020).…”

Section: Assessments With Immersive Learning Technologiesmentioning

confidence: 99%

Game-based assessment framework for virtual reality, augmented reality and digital game-based learning

Udeozor

Chan

Abegão

et al. 2023

Int J Educ Technol High Educ

View full text Add to dashboard Cite

Immersive learning technologies such as virtual reality (VR), augmented reality (AR) and educational digital games offer many benefits to teaching and learning. With their potential to immerse learners in realistic environments and facilitate higher-order cognitive learning, these technologies could be used to complement current classroom pedagogical practices. However, given that these learning environments differ from conventional classroom learning activities, current assessment practices may be insufficient for assessing learning in immersive environments. This paper develops the concept of a game-based assessment framework (GBAF) for educators interested in the assessment of learning in digital games, VR or AR. Importantly, this paper also presents the application of the framework to the design and implementation of assessments for a VR game during the game design phase. Grounded in the principles of Constructive Alignment and the Evidence-Centred Design (ECD) framework, this assessment framework describes the steps to consider for assessments and outlines the components that must be aligned for the design of assessments. To illustrate the application of the GBAF to the design of assessments for immersive learning environments, a stepwise design of assessments for a VR game is presented. The results of the outcome of the assessment of laboratory health and safety competencies of six engineering students is also presented. The GBAF offers simple and useful guidelines for the design of assessments around game tasks. It could serve as a structured basis for educators and researchers to design assessments to measure lower and higher-order cognitive learning in complex immersive environments.

show abstract

Section: Assessments With Immersive Learning Technologiesmentioning

confidence: 99%

Game-based assessment framework for virtual reality, augmented reality and digital game-based learning

Udeozor

Chan

Abegão

et al. 2023

Int J Educ Technol High Educ

View full text Add to dashboard Cite

show abstract

“…The presentation and assembly models specify how the assessment tasks are presented to students and how many tasks will be presented respectively. The ECD has been used in a few studies to design embedded game-based assessments (see Jaffal & Wloka, 2015;Shute, Ke, & Wang, 2017;Shute & Rahimi, 2021). However, the design process is considered complex and time-consuming, and requires advanced statistical and machine-learning skills (Kim et al, 2016;Westera, 2019;Westera et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Measuring learning in digital games: Applying a game‐based assessment framework

Udeozor,

Abegão,

Glassey

2023

Brit J Educational Tech

View full text Add to dashboard Cite

Digital games (DGs) have the potential to immerse learners in simulated real‐world environments that foster contextualised and active learning experiences. These also offer opportunities for performance assessments by providing an environment for students to carry out tasks requiring the application of knowledge and skills learned in the classroom. Although widely used for teaching and learning, there are challenges in using digital game applications for assessment. Designing assessments around tasks in complex learning environments like DGs is not always easy, particularly for educators new to using these technologies for teaching. This paper presents a game‐based assessment framework (GBAF) that is designed to be educator‐friendly and useful for the design of assessments for immersive learning environments. The application of the framework to the design of embedded and external (multiple‐choice questions) assessments is also presented. The GBAF offered an easy and structured basis for the design of two distinct assessments for a computer game. It provides educators with useful guidelines for designing assessments around pre‐existing games. Practitioner notesWhat is already known about this topicImmersive technologies are increasingly being used for teaching and learning due to their benefits to knowledge and skills acquisition. However, for educators interested in using these tools for formal classroom teaching, there is a lack of resources and guidance on assessment designs and implementations. The most commonly used assessment design framework, the evidence‐centred design (ECD), is complicated to use, requiring advanced machine learning skills to implement. This has so far limited its use to large‐scale computerised and game‐based testing.What this paper addsThis paper presents a framework developed to guide educators through the design and implementation of assessments when teaching with immersive learning applications. Grounded in the constructive alignment principles and the ECD framework, the game‐based assessment framework is designed to help educators align assessment tasks to game tasks and the intended learning outcomes. It also provides step‐by‐step guidelines for the identification of appropriate immersive applications. It shows the practical application of the framework to the design of embedded (in‐game) assessments and external (multiple‐choice tests) assessments.Implications for practice and/or policyThe framework presented in this paper provides a structured and pedagogically sound basis for the design of assessments when measuring performance, or skills and knowledge acquisition with immersive learning applications.

show abstract

A validation study of the leadership behaviour assessment.

Heslop¹

2021

View full text Add to dashboard Cite

Employing game analytics techniques in the psychometric measurement of game-based assessments with dynamic content

Cited by 4 publications

References 22 publications

Game-based assessment framework for virtual reality, augmented reality and digital game-based learning

Game-based assessment framework for virtual reality, augmented reality and digital game-based learning

Measuring learning in digital games: Applying a game‐based assessment framework

A validation study of the leadership behaviour assessment.

Contact Info

Product

Resources

About