Personalized mental stress detection with self-organizing map: From laboratory to the field

Tervonen, Jaakko; Puttonen, Sampsa; Sillanpää, Mikko; Hopsu, Leila; Homorodi, Zsolt; Keränen, Janne; Pajukanta, Janne; Tolonen, Antti; Lämsä, Arttu; Mäntyjärvi, Jani

doi:10.1016/j.compbiomed.2020.103935

Cited by 26 publications

(12 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the results reported by Schmidt et al in either the single or multi modality case are lower than the performance of our proposed CNN models (78% F1-score) and the gradient tree boosting algorithm (79% F1score). In addition, the other methods that are evaluated on the WESAD dataset with LOOCV reported lower F1-scores [34], [35]. Tervonen et al [34] proposed a binary stress classification using the self-organizing map with different personalized levels.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Modal Physiological Data Fusion for Affect Estimation Using Deep Learning

Hssayeni

Ghoraani

2021

IEEE Access

View full text Add to dashboard Cite

Automated momentary estimation of positive and negative affects (PA and NA), the basic sense of feeling, can play an essential role in detecting the early signs of mood disorders. Physiological wearable sensors and machine learning have a potential to make such automated and continuous measurements. However, the physiological signals' features that are associated with the subject-reported PA or NA may not be known. In this work, we use data-driven feature extraction based on deep learning to investigate the application of raw physiological signals for estimating PA and NA. Specifically, we propose two multimodal data fusion methods with deep Convolutional Neural Networks. We use the proposed architecture to estimate PA and NA and also classify baseline, stress, and amusement emotions. The training and evaluation of the methods are performed using four physiological and one chest motion signal modalities collected using a chest sensing unit from 15 subjects. Overall, our proposed model performed better than traditional machine learning on hand-crafted features. Utilizing only two modalities, our proposed model estimated PA with a correlation of 0.69 (p<0.05) vs. 0.59 (p<0.05) with traditional machine learning. These correlations were 0.79 (p<0.05) vs. 0.73 (p<0.05) for NA estimation. The best emotion classification was achieved by the traditional method with 79% F1-score and 80% accuracy when all the four physiological modalities are used. This is while with only two modalities, the deep learning achieved 78% F1-score and 79% accuracy.

show abstract

Section: Discussionmentioning

confidence: 99%

“…In addition, the other methods that are evaluated on the WESAD dataset with LOOCV reported lower F1-scores [34], [35]. Tervonen et al [34] proposed a binary stress classification using the self-organizing map with different personalized levels. F1-score of their semi-personal model evaluated using LOOCV on a field dataset was 62%.…”

Section: Discussionmentioning

confidence: 99%

Multi-Modal Physiological Data Fusion for Affect Estimation Using Deep Learning

Hssayeni

Ghoraani

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The features were normalized using within-subject standardization, meaning that each feature was transformed by subtracting its mean and dividing by its standard deviation separately for each participant. Person-specific standardization was conducted instead of person-independent standardization, since it has shown improved performance in earlier work in similar contexts [14,20,37].…”

Section: Data Preprocessing Segmentation and Feature Extractionmentioning

confidence: 99%

Ultra-Short Window Length and Feature Importance Analysis for Cognitive Load Detection from Wearable Sensors

2021

Self Cite

View full text Add to dashboard Cite

Human cognitive capabilities are under constant pressure in the modern information society. Cognitive load detection would be beneficial in several applications of human–computer interaction, including attention management and user interface adaptation. However, current research into accurate and real-time biosignal-based cognitive load detection lacks understanding of the optimal and minimal window length in data segmentation which would allow for more timely, continuous state detection. This study presents a comparative analysis of ultra-short (30 s or less) window lengths in cognitive load detection with a wearable device. Heart rate, heart rate variability, galvanic skin response, and skin temperature features are extracted at six different window lengths and used to train an Extreme Gradient Boosting classifier to detect between cognitive load and rest. A 25 s window showed the highest accury (67.6%), which is similar to earlier studies using the same dataset. Overall, model accuracy tended to decrease as the window length decreased, and lowest performance (60.0%) was observed with a 5 s window. The contribution of different physiological features to the classification performance and the most useful features that react in short windows are also discussed. The analysis provides a promising basis for future real-time applications with wearable sensors.

show abstract

“…Previous studies have collected behavioral data from computer, keystroke, and mouse dynamics such as interaction time, typing pressure, and mouse clicking, but mostly under laboratory conditions [21][22][23][24][25]. The majority of real-life stress-detection studies have been based on behavioral data gathered from smartphones [26][27][28][29][30][31][32]. The behavioral data collection approaches are convenient because they do not require any additional gadgets, but using data from personal devices can pose privacy concerns.…”

Section: Unobtrusive Stress Detectionmentioning

confidence: 99%

Unobtrusive Continuous Stress Detection in Knowledge Work—Statistical Analysis on User Acceptance

et al. 2021

View full text Add to dashboard Cite

Modern knowledge work is highly intense and demanding, exposing workers to long-term psychosocial stress. In order to address the problem, stress detection technologies have been developed, enabling the continuous assessment of personal stress based on multimodal sensor data. However, stakeholders lack insights into how employees perceive different monitoring technologies and whether they are willing to share stress-indicative data in order to sustain well-being at the individual, team, and organizational levels in the knowledge work context. To fill this research gap, we developed a theoretical model for knowledge workers’ interest in sharing their stress-indicative data collected with unobtrusive sensors and examined it empirically using structural equation modeling (SEM) with a survey of 181 European knowledge workers. The results did not show statistically significant privacy concerns regarding environmental sensors such as air quality, sound level, and motion sensors. On the other hand, concerns about more privacy-sensitive methods such as tracking personal device usage patterns did not prevent user acceptance nor intent to share data. Overall, knowledge workers were highly interested in employing stress monitoring technologies to measure their stress levels and receive information about their personal well-being. The results validate the willingness to accept the unobtrusive, continuous stress detection in the context of knowledge work.

show abstract

Personalized mental stress detection with self-organizing map: From laboratory to the field

Cited by 26 publications

References 21 publications

Multi-Modal Physiological Data Fusion for Affect Estimation Using Deep Learning

Multi-Modal Physiological Data Fusion for Affect Estimation Using Deep Learning

Ultra-Short Window Length and Feature Importance Analysis for Cognitive Load Detection from Wearable Sensors

Unobtrusive Continuous Stress Detection in Knowledge Work—Statistical Analysis on User Acceptance

Contact Info

Product

Resources

About