Reliability of MUSE 2 and Tobii Pro Nano at capturing mobile application users' real-time cognitive workload changes

Abstract

IntroductionDespite the importance of cognitive workload in examining the usability of smartphone applications and the popularity of smartphone usage globally, cognitive workload as one attribute of usability tends to be overlooked in Human-Computer Interaction (HCI) studies. Moreover, limited studies that have examined the cognitive workload aspect often measured some summative workloads using subjective measures (e.g., questionnaires). A significant limitation of subjective measures is that they can only assess the overall, subject-perceived cognitive workload after the procedures/tasks have been completed. Such measurements do not reflect the real-time workload fluctuation during the procedures. The reliability of some devices on a smartphone setting has not been thoroughly evaluated.MethodsThis study used mixed methods to empirically study the reliability of an eye-tracking device (i.e., Tobii Pro Nano) and a low-cost electroencephalogram (EEG) device (i.e., MUSE 2) for detecting real-time cognitive workload changes during N-back tasks.ResultsResults suggest that the EEG measurements collected by MUSE 2 are not very useful as indicators of cognitive workload changes in our setting, eye movement measurements collected by Tobii Pro Nano with mobile testing accessory are useful for monitoring cognitive workload fluctuations and tracking down interface design issues in a smartphone setting, and more specifically, the maximum pupil diameter is the preeminent indicator of cognitive workload surges.DiscussionIn conclusion, the pupil diameter measure combined with other subjective ratings would provide a comprehensive user experience assessment of mobile applications. They can also be used to verify the successfulness of a user interface design solution in improving user experience.

Measuring cognitive load in augmented reality with physiological methods: A systematic review

“…For example, the reliability of devices such as Empatica E4 and Muse, which were used in the reviewed studies (Antoniou et al, 2020;Brunzini et al, 2022), is questionable (Borrego et al, 2019;Karwowski et al, 2022).…”

“…One of them is a choice of measuring device and analysis algorithm. For example, the reliability of devices such as Empatica E4 and Muse, which were used in the reviewed studies (Antoniou et al, 2020; Brunzini et al, 2022), is questionable (Borrego et al, 2019; Karwowski et al, 2022). The choice of EEG channels also presents uncertain reliability.…”

An Overview of Approaches and Methods for the Cognitive Workload Estimation in Human–Machine Interaction Scenarios through Wearables Sensors

“…For instance, Bailey & Iqbal showcased the potential of eye tracking in monitoring CW fluctuations during interactive tasks through pupil dilation [67]. Similarly, Zhang & Cui validated the reliability of the Tobii Pro Nano eye tracker in detecting CW changes in real-time during N-back tasks [68]. Wu et al, further, demonstrated the predictive capability of eye-tracking metrics in estimating CW levels with an accuracy of 84.7% during robotic surgical training [69].…”

Introducing a New Mobile Electroencephalography System and Evaluating Its Quality in Comparison to Clinical Electroencephalography

“…Notably, DreamMachine stands out for its affordability, with a manufacturing cost below 150 euros (for the time needed to design and produce the board). In the mobile EEG market, OpenBCI and Muse 2 have emerged as prominent choices, offering cost-effective solutions without compromising performance [9][10][11]. While other mobile EEG devices like MindWave Mobile2 [12], Emotiv EPOC+ [13], and the unicorn hybrid black [14] are recognized for their advanced features and higher price points, Muse 2 and OpenBCI provide more accessible alternatives [15,16].…”

“…While other mobile EEG devices like MindWave Mobile2 [12], Emotiv EPOC+ [13], and the unicorn hybrid black [14] are recognized for their advanced features and higher price points, Muse 2 and OpenBCI provide more accessible alternatives [15,16]. However, it is important to acknowledge the limitations of these devices, including a limited number of channels-16 channels for the Cyton+Daisy board from OpenBCI [17] and 4 channels for the Muse 2 [11]. A higher cost of $1999.00 USD for the OpenBCI system and 249 USD for the Muse 2, along with potential challenges in terms of accessibility, which may pose constraints for certain student and research budgets, are noted.…”