Emotion-recognition using smart watch accelerometer data

Quiroz, Juan C.; Yong, Min Hooi; Geangu, Elena

doi:10.1145/3123024.3125614

Cited by 33 publications

(16 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our results show that personal models outperformed personal baselines and achieved median accuracies higher than 78% for all conditions of the design study for binary classification of happiness versus sadness. This paper is an extended version of preliminary findings published [16]. …”

Section: Introductionmentioning

confidence: 99%

Emotion Recognition Using Smart Watch Sensor Data: Mixed-Design Study

2018

Self Cite

View full text Add to dashboard Cite

BackgroundResearch in psychology has shown that the way a person walks reflects that person’s current mood (or emotional state). Recent studies have used mobile phones to detect emotional states from movement data.ObjectiveThe objective of our study was to investigate the use of movement sensor data from a smart watch to infer an individual’s emotional state. We present our findings of a user study with 50 participants.MethodsThe experimental design is a mixed-design study: within-subjects (emotions: happy, sad, and neutral) and between-subjects (stimulus type: audiovisual “movie clips” and audio “music clips”). Each participant experienced both emotions in a single stimulus type. All participants walked 250 m while wearing a smart watch on one wrist and a heart rate monitor strap on the chest. They also had to answer a short questionnaire (20 items; Positive Affect and Negative Affect Schedule, PANAS) before and after experiencing each emotion. The data obtained from the heart rate monitor served as supplementary information to our data. We performed time series analysis on data from the smart watch and a t test on questionnaire items to measure the change in emotional state. Heart rate data was analyzed using one-way analysis of variance. We extracted features from the time series using sliding windows and used features to train and validate classifiers that determined an individual’s emotion.ResultsOverall, 50 young adults participated in our study; of them, 49 were included for the affective PANAS questionnaire and 44 for the feature extraction and building of personal models. Participants reported feeling less negative affect after watching sad videos or after listening to sad music, P<.006. For the task of emotion recognition using classifiers, our results showed that personal models outperformed personal baselines and achieved median accuracies higher than 78% for all conditions of the design study for binary classification of happiness versus sadness.ConclusionsOur findings show that we are able to detect changes in the emotional state as well as in behavioral responses with data obtained from the smartwatch. Together with high accuracies achieved across all users for classification of happy versus sad emotional states, this is further evidence for the hypothesis that movement sensor data can be used for emotion recognition.

show abstract

Section: Introductionmentioning

confidence: 99%

Emotion Recognition Using Smart Watch Sensor Data: Mixed-Design Study

2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several time series and statistical methods were adopted to analyze changes in mood. It was found that the accuracy of the individual's emotional recognition model was higher than the individual's baseline level, and the classification accuracy for happiness and sadness was higher than 78% [10]. Pollreisz et al used a smart watch to collect data on electrodermal activity (EDA), skin temperature (SKT), and heart rate (HR) for ten subjects.…”

Section: Introductionmentioning

confidence: 99%

Wearable Emotion Recognition Using Heart Rate Data from a Smart Bracelet

Shu

Chen

et al. 2020

Sensors

118

View full text Add to dashboard Cite

Emotion recognition and monitoring based on commonly used wearable devices can play an important role in psychological health monitoring and human-computer interaction. However, the existing methods cannot rely on the common smart bracelets or watches for emotion monitoring in daily life. To address this issue, our study proposes a method for emotional recognition using heart rate data from a wearable smart bracelet. A ‘neutral + target’ pair emotion stimulation experimental paradigm was presented, and a dataset of heart rate from 25 subjects was established, where neutral plus target emotion (neutral, happy, and sad) stimulation video pairs from China’s standard Emotional Video Stimuli materials (CEVS) were applied to the recruited subjects. Normalized features from the data of target emotions normalized by the baseline data of neutral mood were adopted. Emotion recognition experiment results approved the effectiveness of ‘neutral + target’ video pair simulation experimental paradigm, the baseline setting using neutral mood data, and the normalized features, as well as the classifiers of Adaboost and GBDT on this dataset. This method will promote the development of wearable consumer electronic devices for monitoring human emotional moods.

show abstract

“…Experimental Setup. Throughout the experiments, we adopt the 10-fold cross-validation protocol, which is a standard protocol applied to HAR based on sensors [23], [4], [24]. To report the results, we use the recall metric [25], which we also refer to as recognition rate.…”

Section: Resultsmentioning

confidence: 99%

Latent HyperNet: Exploring the Layers of Convolutional Neural Networks

Jordão

Kloss

Schwartz

2018

2018 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Since Convolutional Neural Networks (ConvNets) are able to simultaneously learn features and classifiers to discriminate different categories of activities, recent works have employed ConvNets approaches to perform human activity recognition (HAR) based on wearable sensors, allowing the removal of expensive human work and expert knowledge. However, these approaches have their power of discrimination limited mainly by the large number of parameters that compose the network and the reduced number of samples available for training. Inspired by this, we propose an accurate and robust approach, referred to as Latent HyperNet (LHN). The LHN uses feature maps from early layers (hyper) and projects them, individually, onto a low dimensionality (latent) space. Then, these latent features are concatenated and presented to a classifier. To demonstrate the robustness and accuracy of the LHN, we evaluate it using four different network architectures in five publicly available HAR datasets based on wearable sensors, which vary in the sampling rate and number of activities. We experimentally demonstrate that the proposed LHN is able to capture rich information, improving the results regarding the original ConvNets. Furthermore, the method outperforms existing state-of-the-art methods, on average, by 5.1 percentage points.

show abstract

Emotion-recognition using smart watch accelerometer data

Cited by 33 publications

References 17 publications

Emotion Recognition Using Smart Watch Sensor Data: Mixed-Design Study

Emotion Recognition Using Smart Watch Sensor Data: Mixed-Design Study

Wearable Emotion Recognition Using Heart Rate Data from a Smart Bracelet

Latent HyperNet: Exploring the Layers of Convolutional Neural Networks

Contact Info

Product

Resources

About