Combining think-aloud and physiological data to understand video game experiences

Abstract

Think-aloud protocols are commonly used to evaluate player experiences of video games but suffer from a lack of objectivity and timeliness. On the other hand, quantitative captures of physiological data are effective; providing detailed, unbiased and continuous responses of players, but lack contexts for interpretation. This paper documents how both approaches could be used together in practice by comparing video-cued retrospective think-aloud data and physiological data collected during a video gameplay experiment. We observed that many interesting physiological responses did not feature in participants' think-aloud data, and conversely, reports of interesting experiences were sometimes not observed in the collected physiological data. Through learnings from our experiment, we present some of the challenges when combining these approaches and offer some guidelines as to how qualitative and quantitative data can be used together to gain deeper insights into player experiences.

A Mixed Method Approach for Evaluating and Improving the Design of Learning in Puzzle Games

“…Due to time and resource limitations, user testing often involves relatively small numbers of participants. Observations are typically supplemented with "think-aloud" data (where players are asked to verbalise what they are thinking during play), interviews, questionnaires [21,33] and/or physiological measurement [30,41].…”

Smart Homes that Monitor Breathing and Heart Rate

“…The HCI community has significant literature on the use of physiological sensing for various applications [38,30,9,23]. In particular, HCI researchers have used physiological sensing to evaluate user experience including emotional reactions, stress levels, cognitive performance, and user engagement.…”

Deconstructing Immersion in the Experience Economy Framework for Immersive Dining Experiences through Mixed Reality

“…RTAP was favored over concurrent-think-aloud, as having participants verbalize their thoughts and emotions while experiencing the narrative content might be disruptive to their experience. Furthermore, participants might find it difficult to accurately articulate their experiences whilst also having to engage with the HoloLens 2 content [ 37 ]. Retrospective verbal reporting allows participants to be reflective of their choices and thoughts during the process of using the device, which in turn allows them to point out higher-level causes for individual usability problems.…”

“…While RTAP offers insight into the participants’ mental processes and immersion effects during the HoloLens 2 content narrative, it lacks objectivity and timeliness [ 37 ]. Unbiased, quantitative, detailed, and objective data are effectively captured by biometric measures such as the GSR measured by the Empatica E4 wristbands.…”