K-EmoPhone: A Mobile and Wearable Dataset with In-Situ Emotion, Stress, and Attention Labels

Kang, Soowon; Choi, Woohyeok; Park, Cheul Young; Cha, Narae; Kim, Auk; Khandoker, Ahsan H.; Hadjileontiadis, Leontios; Kim, Heepyung; Jeong, Yong Hoon; Lee, Uichin

doi:10.1038/s41597-023-02248-2

Cited by 9 publications

(3 citation statements)

References 82 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Emotions are complex aspects of the human experience 1 that are recognized, analyzed, and comprehended in the field of Affective Computing (AC) 2 . Recent research in AC has been frequently addressing influence of context in which emotions arise as a determining factor 3 .…”

Section: Background and Summarymentioning

confidence: 99%

See 1 more Smart Citation

EmoWear: Wearable Physiological and Motion Dataset for Emotion Recognition and Context Awareness

Rahmani,

Symons,

Sobhani

et al. 2024

Sci Data

View full text Add to dashboard Cite

The EmoWear dataset provides a bridge to explore Emotion Recognition (ER) via Seismocardiography (SCG), the measurement of small cardio-respiratory induced vibrations on the chest wall through Inertial Measurement Units (IMUs). We recorded Accelerometer (ACC), Gyroscope (GYRO), Electrocardiography (ECG), Blood Volume Pulse (BVP), Respiration (RSP), Electrodermal Activity (EDA), and Skin Temperature (SKT) data from 49 participants who watched validated emotionally stimulating video clips. They self-assessed their emotional valence, arousal, and dominance, as well as extra questions about the video clips. Also, we asked the participants to walk, talk, and drink, so that researchers can detect gait, voice, and swallowing using the same IMU. We demonstrate the effectiveness of emotion stimulation with statistical methods and verify the quality of the collected signals through signal-to-noise ratio and correlation analysis. EmoWear can be used for ER via SCG, ER during gait, multi-modal ER, and the study of IMUs for context-awareness. Targeted contextual information include emotions, gait, voice activity, and drinking, all having the potential to be sensed via a single IMU.

show abstract

Section: Background and Summarymentioning

confidence: 99%

“…Among AC datasets that incorporate IMU sensor data, none are made for investigation of SCG in the field of AC. BIRAFFE2 25 offers ACC data collected from a gamepad; K-EmoCon 36 , K-EmoPhone 2 , and Schmidt et al . 32 collect their ACC data only from the wrist.…”

Section: Background and Summarymentioning

confidence: 99%

EmoWear: Wearable Physiological and Motion Dataset for Emotion Recognition and Context Awareness

Rahmani,

Symons,

Sobhani

et al. 2024

Sci Data

View full text Add to dashboard Cite

show abstract

“…1) Collection Procedure: The data in the K-EmoPhone [22] dataset were collected from 79 participants who used Android phones over the course of one week. The participants were educated through a session explaining details pertaining to the data collection and equipment.…”

Section: B Data Collectionmentioning

confidence: 99%

Design of Contextual Filtered Features for Better Smartphone-User Receptivity Prediction

Alikhanov,

Zhang,

Noh

et al. 2024

IEEE Internet Things J.

View full text Add to dashboard Cite

For the successful engagement of users with incoming interventions, the delivery of Just-In-Time (or opportune moment-OM) interventions is of the utmost importance. This can be accomplished with a machine learning model that utilizes user context data. Smartwatch and smartphone interactions further open the space for further improvement by considering the user's state and environmental information. In this work, a novel feature engineering approach for predicting smartphone user receptivity to Just-In-Time (JIT) interventions is proposed. Proposed approach utilizes rich information from user-smartphone interactions and smartwatch sensor data to extract contextually filtered features. The superiority of the proposed feature engineering method is demonstrated by developing a machine learning model that predicts a participant's receptivity prior to administering an Experience Sampling Method (ESM) questionnaire. The proposed approach is evaluated using the KEmoPhone dataset, which contains 3,334 ESM answers collected over the course of one week from 73 participants. To reduce the bias that may result in selecting specific classifier, multiple classifiers are tested and their average ROC-AUC metrics are compared. The results show that the proposed feature engineering method -CFF improves the prediction performance, achieving an ROC-AUC of 56.5% as opposed to 54.7% obtained using conventional features (statistical moments of time series over an 80 min window). Such superior performance is consistent across multiple machine learning classification algorithms. Full research code will be released in jupyter notebooks for facilitating the research in the domain at https://github.com/Jumabek/receptivity upon the publication of this research work.

show abstract

Multimodal emotion recognition: A comprehensive review, trends, and challenges

Ramaswamy,

Palaniswamy

2024

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Automatic emotion recognition is a burgeoning field of research and has its roots in psychology and cognitive science. This article comprehensively reviews multimodal emotion recognition, covering various aspects such as emotion theories, discrete and dimensional models, emotional response systems, datasets, and current trends. This article reviewed 179 multimodal emotion recognition literature papers from 2017 to 2023 to reflect on the current trends in multimodal affective computing. This article covers various modalities used in emotion recognition based on the emotional response system under four categories: subjective experience comprising text and self‐report; peripheral physiology comprising electrodermal, cardiovascular, facial muscle, and respiration activity; central physiology comprising EEG, neuroimaging, and EOG; behavior comprising facial, vocal, whole‐body behavior, and observer ratings. This review summarizes the measures and behavior of each modality under various emotional states. This article provides an extensive list of multimodal datasets and their unique characteristics. The recent advances in multimodal emotion recognition are grouped based on the research focus areas such as emotion elicitation strategy, data collection and handling, the impact of culture and modality on multimodal emotion recognition systems, feature extraction, feature selection, alignment of signals across the modalities, and fusion strategies. The recent multimodal fusion strategies are detailed in this article, as extracting shared representations of different modalities, removing redundant features from different modalities, and learning critical features from each modality are crucial for multimodal emotion recognition. This article summarizes the strengths and weaknesses of multimodal emotion recognition based on the review outcome, along with challenges and future work in multimodal emotion recognition. This article aims to serve as a lucid introduction, covering all aspects of multimodal emotion recognition for novices.This article is categorized under: Fundamental Concepts of Data and Knowledge > Human Centricity and User Interaction Technologies > Cognitive Computing Technologies > Artificial Intelligence

show abstract

K-EmoPhone: A Mobile and Wearable Dataset with In-Situ Emotion, Stress, and Attention Labels

Cited by 9 publications

References 82 publications

EmoWear: Wearable Physiological and Motion Dataset for Emotion Recognition and Context Awareness

EmoWear: Wearable Physiological and Motion Dataset for Emotion Recognition and Context Awareness

Design of Contextual Filtered Features for Better Smartphone-User Receptivity Prediction

Multimodal emotion recognition: A comprehensive review, trends, and challenges

Contact Info

Product

Resources

About