DeepFlow: Detecting Optimal User Experience From Physiological Data Using Deep Neural Networks

Maier, M. E.; Elsner, Daniel; Marouane, Chadly; Zehnle, Meike; Fuchs, Christoph

doi:10.24963/ijcai.2019/196

Cited by 48 publications

(39 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The low performance in the easy difficulty could be explained by a bias of the classifier toward the anxiety class which is already observed in the confusion matrix computed on the laboratory data (see Table 2). Interestingly a similar bias was reported in [22] who showed that boredom is more difficult to detect than flow and stress. However this bias might not be the only responsible for the drop of performance.…”

Section: Model Performancesupporting

confidence: 69%

“…Although the best reported accuracy of 73% (for two classes) is lower than the 78% (for three classes) reported in [19], our model is user-independent contrarily to the one proposed in [19]. The closest study to our work is probably [22] as anxiety, boredom and flow were detected using a deep architecture trained on 15.5 hours of physiological signals (heart rate and EDA). A very similar performance was obtained using this approach (around 70% to classify 2 classes).…”

Section: Model Performancementioning

confidence: 79%

“…The current paper is a significant improvement of this work, which in particular now uses deep networks to improve performance and evaluates players perception of affective difficulty adaptation. A deep convolutional network was used in [22] to detect flow, boredom, and stress from heart rate and electrodermal activity during Tetris play (15.5 hours of data). The results demonstrate that user-indepedant models can achieve an accuracy around 70% when classifying two classes (stress vs. not stressed and flow vs. not flow).…”

Section: Affective Modeling In Gamesmentioning

confidence: 99%

See 2 more Smart Citations

User Evaluation of Affective Dynamic Difficulty Adjustment Based on Physiological Deep Learning

Chanel

Lopes

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Challenging players is a fundamental element when designing games, but finding the perfect balance between a frustrating and boring experience can become a challenge in itself. This paper proposes the usage of deep data-driven affective models capable of detecting anxiety and boredom from raw human physiological signals to adapt the fluctuation of difficulty in a game of Tetris. The first phase of this work was to construct several emotion detection models for performance comparison, where the most accurate model achieved an average accuracy of 73.2%. A user evaluation was subsequently conducted on a total of 56 participants to validate the efficiency of the most accurate model obtained using two diverging difficulty adaptation types. One method adapts difficulty according to raw values directly outputted by the affective model (ABS), while another compares the current value with the previous one to influence difficulty (REA). Results show that the model using the ABS adaptation was capable of effectively adjusting the overall difficulty based on a player's physiological state during a game of Tetris.

show abstract

Section: Model Performancesupporting

confidence: 69%

Section: Model Performancementioning

confidence: 79%

Section: Affective Modeling In Gamesmentioning

confidence: 99%

See 1 more Smart Citation

User Evaluation of Affective Dynamic Difficulty Adjustment Based on Physiological Deep Learning

Chanel

Lopes

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…End-to-end DL was utilized in the flow state recognition task, which was defined as ”affective state of optimal experience, total immersion and high productivity” [ 36 ]. Based on BVP and EDA signals from Emaptica E4, a model was trained to recognize two (high/low flow level) or three flow states (boredom, stress, and flow).…”

Section: Related Workmentioning

confidence: 99%

Can We Ditch Feature Engineering? End-to-End Deep Learning for Affect Recognition from Physiological Sensor Data

Dzieżyc

Gjoreski

Kazienko

et al. 2020

Sensors

View full text Add to dashboard Cite

To further extend the applicability of wearable sensors in various domains such as mobile health systems and the automotive industry, new methods for accurately extracting subtle physiological information from these wearable sensors are required. However, the extraction of valuable information from physiological signals is still challenging—smartphones can count steps and compute heart rate, but they cannot recognize emotions and related affective states. This study analyzes the possibility of using end-to-end multimodal deep learning (DL) methods for affect recognition. Ten end-to-end DL architectures are compared on four different datasets with diverse raw physiological signals used for affect recognition, including emotional and stress states. The DL architectures specialized for time-series classification were enhanced to simultaneously facilitate learning from multiple sensors, each having their own sampling frequency. To enable fair comparison among the different DL architectures, Bayesian optimization was used for hyperparameter tuning. The experimental results showed that the performance of the models depends on the intensity of the physiological response induced by the affective stimuli, i.e., the DL models recognize stress induced by the Trier Social Stress Test more successfully than they recognize emotional changes induced by watching affective content, e.g., funny videos. Additionally, the results showed that the CNN-based architectures might be more suitable than LSTM-based architectures for affect recognition from physiological sensors.

show abstract

“…In [15] psychophysiological signal quality estimators were proposed that were utilized to affect recognition systems. Further, in [16] findings in the domain of estimations of affective states of users' optimal experiences were presented. Estimated signals through end-to-end intelligent architecture possessed 67.5% accuracy in recognizing different affective states, including stress.…”

Section: Introduction and Literature Reviewmentioning

confidence: 99%

Semi-supervised Deep Learning for Stress Prediction: A Review and Novel Solutions

Alshamrani¹

2021

IJACSA

View full text Add to dashboard Cite

This research introduces a novel self-supervised deep learning model for stress detection using an intelligent solution that detects the stress state using the physiological parameters. The first part of this research represents a concise review of different intelligent techniques for processing physiological data and the emotional states of humans. Also, for all covered methods, special attention is made to semi-supervised learning algorithms. In the second part of the paper, a novel semi-supervised deep learning model for predicting the stress state is proposed. It is the first attempt of using contrastive learning for the stress prediction tasks. The model is based on utilizing generative and contrastive features specially tailored for treating time-series data. A widely popular multimodal WESAD (Wearable Stress and Affect Detection) data set is exploited for experimental purposes. It consists of physiological and motion data recorded from the wrist and chest-worn devices. To provide an intelligent solution that will be widely applicable, only the wrist data recorded from smartwatches is exploited during the model's training. The proposed model in this research is tested on a single subject's data and predicts the stress and non-stress events. Keeping in mind that the initial data was unbalanced with only 11% of the stress data, data augmentation techniques are applied within the model to provide additional reliable training information. The model shows significant potential in clustering stress conditions, and it presents accuracy in the range with other state-of-the-art solutions. The most significant benefits of using this model are its prediction capabilities when dealing with unlabeled data and performances when undersized data cannot be processed optimally by traditional intelligent methods.

show abstract

DeepFlow: Detecting Optimal User Experience From Physiological Data Using Deep Neural Networks

Cited by 48 publications

References 4 publications

User Evaluation of Affective Dynamic Difficulty Adjustment Based on Physiological Deep Learning

User Evaluation of Affective Dynamic Difficulty Adjustment Based on Physiological Deep Learning

Can We Ditch Feature Engineering? End-to-End Deep Learning for Affect Recognition from Physiological Sensor Data

Semi-supervised Deep Learning for Stress Prediction: A Review and Novel Solutions

Contact Info

Product

Resources

About