K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations

Abstract

Recognizing emotions during social interactions has many potential applications with the popularization of low-cost mobile sensors, but a challenge remains with the lack of naturalistic affective interaction data. Most existing emotion datasets do not support studying idiosyncratic emotions arising in the wild as they were collected in constrained environments. Therefore, studying emotions in the context of social interactions requires a novel dataset, and K-EmoCon is such a multimodal dataset with comprehensive annotations of continuous emotions during naturalistic conversations. The dataset contains multimodal measurements, including audiovisual recordings, EEG, and peripheral physiological signals, acquired with off-the-shelf devices from 16 sessions of approximately 10-minute long paired debates on a social issue. Distinct from previous datasets, it includes emotion annotations from all three available perspectives: self, debate partner, and external observers. Raters annotated emotional displays at intervals of every 5 seconds while viewing the debate footage, in terms of arousal-valence and 18 additional categorical emotions. The resulting K-EmoCon is the first publicly available emotion dataset accommodating the multiperspective assessment of emotions during social interactions.

CorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological Sensors

“…In total, 20 participants (12 m, 8 f) aged between 22 and 32 participated in the data collection experiment of MERCA. The number of participants in MERCA dataset is similar to some of the widely used emotion recognition datasets (e.g., CASE [ 19 ], K-EmoCon [ 92 ], DECAF [ 93 ]) with continuously annotated physiological signals. Participants were recruited from different institutions with diverse backgrounds, education levels and nationalities.…”

Personality-Based Affective Adaptation Methods for Intelligent Systems

“…Unlike two existing datasets, ASCERTAIN [ 31 ] and AMIGOS [ 32 ], which also collect affective data along with the personality assessment, BIRAFFE provides details of changes in stimuli over time in a form of game logs, which facilitates the analysis of subjects emotions in reaction to the complex affective stimuli presented. In addition, this study has several more features that distinguish it from existing affective data sets [ 17 , 26 ], such as: a large sample (206 individuals), the inclusion of conflicting stimuli ( p+s– and p–s+ ), the use of affective games as an experimental environment, and the introduction of new widgets for the self-assessment of emotions by the subjects.…”

“…It is also possible to choose a dataset that has specific signals (e.g., EEG or ECG). The K-EmoCon dataset authors provide the up-to-date overview [ 26 ]. Unfortunately, these datasets suffer from a small research sample [ 26 , 27 ].…”

“…The K-EmoCon dataset authors provide the up-to-date overview [ 26 ]. Unfortunately, these datasets suffer from a small research sample [ 26 , 27 ]. In review [ 26 ], only the SEMAINE study [ 28 ] has more than 100 participants, while the others have an average of 30 (minimum seven, maximum 64).…”

Emotion Recognition From EEG Signal Focusing on Deep Learning and Shallow Learning Techniques

“…7. Emotional photos [60], audio [61]- [63], video [57], [64], audio-visual film clips [43], [58], [65]- [67], or any other emotional task or event [68] are used as stimuli during the recording of the EEG signal. Audio-visual film or movie clips are the best choices as stimuli for emotionrelated study.…”