Artificial neural networks-based classification of emotions using wristband heart rate monitor data

Abstract

Heart rate variability (HRV) is an objective measure of emotional regulation. This study aimed to estimate the accuracy with which an artificial neural network (ANN) algorithm could classify emotions using HRV data that were obtained using wristband heart rate monitors.Four emotions were evoked during gameplay: pleasure, happiness, fear, and anger. Seven normalized HRV features (i.e., 3 time-domain features, 3 frequency-domain features, and heart rate), which yielded 29,727 segments during gameplay, were collected and analyzed first by statistics and then classified by the trained ANN model.General linear model adjusted for individual differences in HRV showed that all HRV features significantly differed across emotions, despite disparities in their magnitudes and associations. When compared to neutral status (i.e., no emotion evoked), the mean of R-R interval was significantly higher for pleasure and fear but lower for happiness and anger. In addition, pleasure evidenced the HRV features that suggested a superior parasympathetic to sympathetic activation. Happiness was associated with a prominent sympathetic activation. These statistical findings suggest that HRV features significantly differ across emotions evoked by gameplay. When further utilizing ANN-based emotion classification, the accuracy rates for prediction were above 75.0% across the 4 emotions with accuracy rates for classification of paired emotions ranging from 82.0% to 93.4%.For classifying emotion in an individual person, the trained ANN model utilizing HRV features yielded a high accuracy rate in our study. ANN is a time-efficient and accurate means to classify emotions using HRV data obtained from wristband heart rate monitors. Thus, this integrated platform can help monitor and quantify human emotions and physiological biometrics.

Assessment of Cognitive Student Engagement Using Heart Rate Data in Distance Learning during COVID-19

“…Heart rate has been a tool widely used for identifying and monitoring emotions, attention, autonomic process and mental conditions [30][31][32][33][34]; heart rate is associated with the autonomic nervous system, which responds to a stimulus or resting states [33,35]. Emotions have three components: cognitive, physiological and behavior [34].…”

“…Cognitive engagement can be determined through the heart rate since the exposure to stimulus or tasks activates neural mechanisms and, consequently, triggers an acceleration or deceleration in the heart rate, which has been an indicator of alertness and drowsiness [36][37][38].The advantages of using heart rate include it is noninvasive, easy, and cheap to get [33,34], experiments with heart rate are simple to set up and can be used in conjunction with other biometric measures like facial expressions and respiration [38,39]. The disadvantages associated with the heart rate are the conditions of the environment under which happen the data collection since they are challenging to eliminate, and also the response time to a stimulus is long; these two points generate more uncertainty [30,40,41].…”

Estimation of Office Worker’s Emotion Types Using Two-dimensional Model Consisted of Biometric Signals

“…The development of technology for estimating the emotions of workers using objective biometric indicators is expected to provide useful information for improving work efficiency and safety, preventing job separation due to work stress, and managing occupational health. In the last decade, several efforts have tried to develop the methods to identify the types of emotion from physiological signals including electroencephalogram (EEG), electrodermal activity, electrocardiogram (ECG), pulse wave, respiration, and eye-movement [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15]. However, there are many issues in estimating emotions in the workplace from these signals.…”

“…Additionally, power spectrum of PRV could differ depending on the measurement site [18]. PRV may provide useful features for estimating emotions [1,3,6], but interpretations of the associations of PRV with emotions cannot be referenced from those of HRV. Finally, the higher the number of signal modality, the better the accuracy of emotion estimation [3,6,10,14,15], but at the expense of ease of use in the workplace.…”

Non-Contact Heart Rate Detection Based on Hand Vein Transillumination Imaging

“…In the medical field, monitoring the heart rate can play a role in preventive care [1,2], such as the screening of patients with early cardiovascular disease. The premise of affective computing is that changes in physiological states such as heart rate are closely linked to people's emotions [3,4], so heart rate parameters are essential for building a comprehensive emotion recognition system. In addition, in recent years, heart rate detection has attracted increasing attention in the field of biometric authentication [5,6], and it can be used as an important parameter to improve the anti-spoofing systems.…”