A Systematic Review for Cognitive State-Based QoE/UX Evaluation

Abstract

Traditional evaluation of user experience is subjective by nature, for what is sought is to use data from physiological and behavioral sensors to interpret the relationship that the user’s cognitive states have with the elements of a graphical interface and interaction mechanisms. This study presents the systematic review that was developed to determine the cognitive states that are being investigated in the context of Quality of Experience (QoE)/User Experience (UX) evaluation, as well as the signals and characteristics obtained, machine learning models used, evaluation architectures proposed, and the results achieved. Twenty-nine papers published in 2014–2019 were selected from eight online sources of information, of which 24% were related to the classification of cognitive states, 17% described evaluation architectures, and 41% presented correlations between different signals, cognitive states, and QoE/UX metrics, among others. The amount of identified studies was low in comparison with cognitive state research in other contexts, such as driving or other critical activities; however, this provides a starting point to analyze and interpret states such as mental workload, confusion, and mental stress from various human signals and propose more robust QoE/UX evaluation architectures.

A Systematic Review of In-Vehicle Physiological Indices and Sensor Technology for Driver Mental Workload Monitoring

“…Similarly, the effect of multiple task handling on performance, as reflected by MWL, has also been studied in air traffic controllers [ 11 ], electric overhead traveling crane operators [ 12 ], and human system interface operators in nuclear power plants [ 13 ]. In the consumer context, MWL has been studied to understand the Quality of Experience (QoE) and User Experience (UX) [ 14 ].…”

Long-Term Video QoE Assessment Studies: A Systematic Review

“…This is probably due to rapid changes and developments of technologies. We found a few examples of the systematic reviews of the Quality of Experience (QoE) or living labs studies [17], [18], [19]. As QoE measures the delight or annoyance of a customer's experiences with a service it is an important aspect, especially for service and network providers.…”