A Transformer Architecture for Stress Detection from ECG

Behinaein, Behnam; Bhatti, Anubhav; Rodenburg, Dirk; Hungler, Paul; Etemad, Ali

doi:10.1145/3460421.3480427

Cited by 40 publications

(20 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Arjun et al [42] employ a variation of the Transformer, the Vision Transformer [43] to process EEG signals for emotion recognition, converting the EEG signals into images using continuous wavelet transform. Behinaein et al [44] propose to detect stress from ECG signals, by using a 1D-CNN followed by a Transformer and a FCN as classifier.…”

Section: Related Workmentioning

confidence: 99%

Transformer-Based Self-Supervised Learning for Emotion Recognition

Vazquez-Rodriguez,

Lefebvre,

Cumin

et al. 2022

Preprint

View full text Add to dashboard Cite

In order to exploit representations of time-series signals, such as physiological signals, it is essential that these representations capture relevant information from the whole signal. In this work, we propose to use a Transformer-based model to process electrocardiograms (ECG) for emotion recognition. Attention mechanisms of the Transformer can be used to build contextualized representations for a signal, giving more importance to relevant parts. These representations may then be processed with a fully-connected network to predict emotions.To overcome the relatively small size of datasets with emotional labels, we employ self-supervised learning. We gathered several ECG datasets with no labels of emotion to pre-train our model, which we then fine-tuned for emotion recognition on the AMIGOS dataset. We show that our approach reaches state-of-the-art performances for emotion recognition using ECG signals on AMIGOS. More generally, our experiments show that transformers and pre-training are promising strategies for emotion recognition with physiological signals.

show abstract

Section: Related Workmentioning

confidence: 99%

Transformer-Based Self-Supervised Learning for Emotion Recognition

Vazquez-Rodriguez,

Lefebvre,

Cumin

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…On this basis, they made arrhythmia classification from ECG signals. Similarly, Behinaein et al [2] placed a convolutional front-end before the transformer encoder to extract more informative representations, and achieved stress detection from ECG signals. Despite models with these structures being able to capture either local spatial dependencies, or temporal dependencies, or global information, they lack the capability to learn both local and global interactions simultaneously.…”

Section: Related Workmentioning

confidence: 99%

“…Inspired by [2,38], the convolutional front-end module consists of two parts (shown in Figure 2). Each part starts with a 1D convolutional layer (1x3 padded convolution), then followed by a batch normalization and a rectified linear unit (ReLU).…”

Section: Convolutional Front-endmentioning

confidence: 99%

“…To compare with the state-of-the-art physiological signals-based emotion recognition approaches, we chose the following methods as baselines: (1) Naive Bayes (NB) [47], (2) Support Vector Machine with linear kernel (SVM-LR) [35], (3) Support Vector Machine with radial basis function kernel (SVM-RBF) [35], (4) eXtreme Gradient Boosting (XGBoost) [47], (5) bidirectional LSTM (BiLSTM) [47]; and (6) transformer [2]. For the first four methods, we extracted both the time-domain and frequency-domain features based on the proposed features in the literature [14].…”

Section: Baselinesmentioning

confidence: 99%

See 1 more Smart Citation

Mobile Emotion Recognition via Multiple Physiological Signals using Convolution-augmented Transformer

Yang

Tag

et al. 2022

Proceedings of the 2022 International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

Recognising and monitoring emotional states play a crucial role in mental health and well-being management. Importantly, with the widespread adoption of smart mobile and wearable devices, it has become easier to collect long-term and granular emotion-related physiological data passively, continuously, and remotely. This creates new opportunities to help individuals manage their emotions and well-being in a less intrusive manner using off-the-shelf low-cost devices. Pervasive emotion recognition based on physiological signals is, however, still challenging due to the difficulty to efficiently extract high-order correlations between physiological signals and users' emotional states. In this paper, we propose a novel end-to-end emotion recognition system based on a convolution-augmented transformer architecture. Specifically, it can recognise users' emotions on the dimensions of arousal and valence by learning both the global and local fine-grained associations and dependencies within and across multimodal physiological data (including blood volume pulse, electrodermal activity, heart rate, and skin temperature). We extensively evaluated the performance of our model using the K-EmoCon dataset, which is acquired in naturalistic conversations using off-the-shelf devices and contains spontaneous emotion data. Our results demonstrate that our approach outperforms the baselines and achieves state-of-the-art or competitive performance. We also demonstrate the effectiveness and generalizability of our system on another affective dataset which used affect inducement and commercial physiological sensors.

show abstract

“…This was followed by transfer learning for downstream supervised classification. Behinaein et al [40] proposed a transformer mechanism to detect stress using ECG signals in two publicly available datasets. In this study, the deep learning network comprises a convolutional subnetwork, a transformer encoder, and a fully connected subnetwork for stress classification.…”

Section: A Uni-modal Affective Computingmentioning

confidence: 99%

Attentive Cross-modal Connections for Deep Multimodal Wearable-based Emotion Recognition

Bhatti

Behinaein

Rodenburg

et al. 2021

2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)

Self Cite

View full text Add to dashboard Cite

We propose cross-modal attentive connections, a new dynamic and effective technique for multimodal representation learning from wearable data. Our solution can be integrated into any stage of the pipeline, i.e., after any convolutional layer or block, to create intermediate connections between individual streams responsible for processing each modality. Additionally, our method benefits from two properties. First, it can share information uni-directionally (from one modality to the other) or bi-directionally. Second, it can be integrated into multiple stages at the same time to further allow network gradients to be exchanged in several touch-points. We perform extensive experiments on three public multimodal wearable datasets, WE-SAD, SWELL-KW, and CASE, and demonstrate that our method can effectively regulate and share information between different modalities to learn better representations. Our experiments further demonstrate that once integrated into simple CNN-based multimodal solutions (2, 3, or 4 modalities), our method can result in superior or competitive performance to state-of-the-art and outperform a variety of baseline uni-modal and classical multimodal methods.

show abstract

A Transformer Architecture for Stress Detection from ECG

Cited by 40 publications

References 19 publications

Transformer-Based Self-Supervised Learning for Emotion Recognition

Transformer-Based Self-Supervised Learning for Emotion Recognition

Mobile Emotion Recognition via Multiple Physiological Signals using Convolution-augmented Transformer

Attentive Cross-modal Connections for Deep Multimodal Wearable-based Emotion Recognition

Contact Info

Product

Resources

About