DEAP: A Database for Emotion Analysis ;Using Physiological Signals

Abstract

Abstract-We present a multimodal data set for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance, and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence, and like/dislike ratings using the modalities of EEG, peripheral physiological signals, and multimedia content analysis. Finally, decision fusion of the classification results from different modalities is performed. The data set is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.

O Uso de Biomarcadores em Pesquisas de Marketing

“…Além da apresentação detalhada dos métodos de coleta e análise, importantes para a replicação do estudo, e de corroborar outros trabalhos que já haviam evidenciado a relação das respostas autonômicas com emoções, os autores desse estudo tornaram pública sua base de dados com 30 sujeitos que avaliaram 40 clipes, conclamando os pesquisadores na busca por métodos e algoritmos que expliquem a relação entre as valências dos videoclipes e as emoções resultantes (Koelstra, Soleymani, Yazdani, & Nijholt, 2012). Em outro estudo, EEG, condutância da pele e da frequência cardíaca foram as medidas utilizadas para aferir a resposta neural decorrente do ato de assistir à televisão naturalmente, em 15 sujeitos, tendo-se evidenciado variações específicas nessas medidas, conforme o conteúdo a que assistiram e com o ato de se lembrar do mesmo ou não (Vecchiato et al, 2010).…”

“…Ferramentas que registram respostas secundárias a comandos neuronais (eye-tracker, condutância elétrica da pele, EMG, controle hormonal, dentre outros), possuem menor tempo e custo de coleta, e podem ser aplicadas de forma simultânea e sincronizada, fato que amplia a capacidade de explicação dos achados (Koelstra et al, 2012). A exploração de emoções básicas via EMG, somadas à condutância elétrica da pele, podem ser úteis na medida de respostas autonômicas na avaliação de diferentes estímulos comerciais, como anúncios audiovisuais, sons, imagens, manipulação de elementos visuais, como cores, palavras, dentre outros.…”

Affective state detection via facial expression analysis within a human–computer interaction context

“…Furthermore, Siddiqi et al had collected a dataset in which the expressions represent real life situations including emulated, semi-naturalistic, and naturalistic expressions (Siddiqi et al 2017). Moreover, another multimodal dataset entitled DEAP, is a dataset for Emotion Analysis using EEG, Physiological and Video Signals for 32 subjects (frontal face video was recorded for only 22), who individually watched 40 1-min music videos of different genres as a stimulus, to induce different affective and emotional states (Koelstra et al 2012). Generally, such datasets could be used for the analysis of affective states, along with the assessment and self-reports made by the subjects themselves, such as using the Self-Assessment Manikin (SAM) scales (Bradley and Lang 1994), to typify the awareness of subjects' feelings.…”

“…These situations include: general operating systems tasks (tagged as OS), online shopping tasks (tagged as Online), and spreadsheet tasks (tagged as Excel), and entertainment task playing Pacman (tagged as Pacman). Secondly, passive interaction that represents situations where the user does not exert effort, which refers to the recordings contained within the DEAP dataset (Koelstra et al 2012), where the subjects passively watched videos on Youtube without explicit interaction.…”

Emotion Understanding Using Multimodal Information Based on Autobiographical Memories for Alzheimer’s Patients

“…Several datasets, focusing on different applications, are available for emotion recognition. For example, DEAP dataset provides EEG and face recordings of participants while they watch musical videos just for the analysis of human affective states [9]; SEMAINE database aims to provide voice and facial information to study the behaviour of subjects interacting with virtual avatars [23]; MAHNOB-HCI database was created for the study of emotions while humans are watching multimedia, supplying several data such as audio, an RGB video and five monochrome videos of the face, EEG, ECG, respiration amplitude, skin temperature and eye-gaze data [21]; or CASMEII dataset which studies facial micro-expressions for security and medical applications, requiring cameras of higher frame rate and spatial resolution [24].…”