Don’t Leave Me Alone: Retrospective Think Aloud Supported by Real-Time Monitoring of Participant’s Physiology

Abstract

Think aloud protocols are widely applied in user experience studies. In this paper, the effect of two different applications of the Retrospective Think Aloud (RTA) protocol on the number of user-reported usability issues is examined. To this end, 30 users were asked to use the National Cadastre and Mapping Agency web application and complete a set of tasks, such as measuring the land area of a square in their hometown. The order of tasks was randomized per participant. Next, participants were involved in RTA sessions. Each participant was involved in two different RTA modes: (a) the strict guidance, in which the facilitator stayed in the background and prompted participants to keep thinking aloud based on his judgement and experience, and (b) the physiology-supported interventions, in which the facilitator intervened based on real-time monitoring of user's physiological signals. During each session, three participant's physiological signals were recorded: skin conductance, skin temperature and blood volume pulse. Participants were also asked to provide valence-arousal ratings for each self-reported usability issue. Analysis of the collected data showed that participants in the physiology-supported RTA mode reported significantly more usability issues. No significant effect of the RTA mode was found on the valence-arousal ratings for the reported usability issues. Participants' physiological signals during the RTA sessions did not also differ significantly between the two modes.

Human-computer interaction

“…The use of an RTA session can highlight more complex and explanatory data than concurrent thinking aloud because participants act under no pressure and they are free to think aloud in a natural way (Eger, Ball, Stevens, & Dodd, 2007). In our study, the physiology-supported intervention mode of RTA was used (Liapis, Katsanos, & Xenos, 2018). In this mode, the facilitator has real-time access to the participant's bio-signals while the participant is thinking aloud.…”

“…In this mode, the facilitator has real-time access to the participant's bio-signals while the participant is thinking aloud. We chose the physiology-supported intervention mode because it has been found to support users in reporting significantly more usability issues compared to the traditional application of RTA (Liapis et al, 2018).…”