Emotion recognition based on 3D fuzzy visual and EEG features in movie clips

Lee, Giyoung; Kwon, Mingu; Sri, Swathi Kavuri; Lee, Minho

doi:10.1016/j.neucom.2014.04.008

Cited by 53 publications

(25 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Feature extraction is so important which can provide the most efficient analysis. The main objective is to obtain reliable data for classification and effective analysis of the signals [23–26]. In this study, wavelet-based feature extraction was applied to the EEG records.…”

Section: Methodsmentioning

confidence: 99%

Wavelet-based study of valence–arousal model of emotions on EEG signals with LabVIEW

Aydin

Güler

2016

Brain Inf.

View full text Add to dashboard Cite

This paper illustrates the wavelet-based feature extraction for emotion assessment using electroencephalogram (EEG) signal through graphical coding design. Two-dimensional (valence–arousal) emotion model was studied. Different emotions (happy, joy, melancholy, and disgust) were studied for assessment. These emotions were stimulated by video clips. EEG signals obtained from four subjects were decomposed into five frequency bands (gamma, beta, alpha, theta, and delta) using “db5” wavelet function. Relative features were calculated to obtain further information. Impact of the emotions according to valence value was observed to be optimal on power spectral density of gamma band. The main objective of this work is not only to investigate the influence of the emotions on different frequency bands but also to overcome the difficulties in the text-based program. This work offers an alternative approach for emotion evaluation through EEG processing. There are a number of methods for emotion recognition such as wavelet transform-based, Fourier transform-based, and Hilbert–Huang transform-based methods. However, the majority of these methods have been applied with the text-based programming languages. In this study, we proposed and implemented an experimental feature extraction with graphics-based language, which provides great convenience in bioelectrical signal processing.

show abstract

Section: Methodsmentioning

confidence: 99%

Wavelet-based study of valence–arousal model of emotions on EEG signals with LabVIEW

Aydin

Güler

2016

Brain Inf.

View full text Add to dashboard Cite

show abstract

“…These proposed systems aim to explore or improve EEG-based emotion recognition systems. [2,39,41,42,49,50,57,61,63,92,104,108,109,117,131,136,149,152,157,173,174,185,186,189,191,[195][196][197][198][199][200][201][202][203][204][205][206][207][208][209]217,219,[223][224][225]229,[262][263][264][265][266]<...>…”

Section: Monitoringmentioning

confidence: 99%

Review and Classification of Emotion Recognition Based on EEG Brain-Computer Interface System Research: A Systematic Review

et al. 2017

View full text Add to dashboard Cite

Abstract:Recent developments and studies in brain-computer interface (BCI) technologies have facilitated emotion detection and classification. Many BCI studies have sought to investigate, detect, and recognize participants' emotional affective states. The applied domains for these studies are varied, and include such fields as communication, education, entertainment, and medicine. To understand trends in electroencephalography (EEG)-based emotion recognition system research and to provide practitioners and researchers with insights into and future directions for emotion recognition systems, this study set out to review published articles on emotion detection, recognition, and classification. The study also reviews current and future trends and discusses how these trends may impact researchers and practitioners alike. We reviewed 285 articles, of which 160 were refereed journal articles that were published since the inception of affective computing research. The articles were classified based on a scheme consisting of two categories: research orientation and domains/applications. Our results show considerable growth of EEG-based emotion detection journal publications. This growth reflects an increased research interest in EEG-based emotion detection as a salient and legitimate research area. Such factors as the proliferation of wireless EEG devices, advances in computational intelligence techniques, and machine learning spurred this growth.

show abstract

“…e visual feature extraction is performed by applying 3D fuzzy GIST [10] based on a tensor data of same dimension that includes the Hue (H)-Saturation (S)-Intensity (I) color space and the scene's dynamic properties in the third dimension. ese tensor data are M × M × T, where T is the duration, or more speci cally the number of frames, of a scene, and M × M is the width and height of a single frame (M 256).…”

Section: Visual Feature Extractionmentioning

confidence: 99%

Emotional Video to Audio Transformation Using Deep Recurrent Neural Networks and a Neuro-Fuzzy System

Sergio

Lee

2020

Mathematical Problems in Engineering

Self Cite

View full text Add to dashboard Cite

Generating music with emotion similar to that of an input video is a very relevant issue nowadays. Video content creators and automatic movie directors bene t from maintaining their viewers engaged, which can be facilitated by producing novel material eliciting stronger emotions in them. Moreover, there is currently a demand for more empathetic computers to aid humans in applications such as augmenting the perception ability of visually-and/or hearing-impaired people. Current approaches overlook the video's emotional characteristics in the music generation step, only consider static images instead of videos, are unable to generate novel music, and require a high level of human e ort and skills. In this study, we propose a novel hybrid deep neural network that uses an Adaptive Neuro-Fuzzy Inference System to predict a video's emotion from its visual features and a deep Long Short-Term Memory Recurrent Neural Network to generate its corresponding audio signals with similar emotional inkling. e former is able to appropriately model emotions due to its fuzzy properties, and the latter is able to model data with dynamic time properties well due to the availability of the previous hidden state information. e novelty of our proposed method lies in the extraction of visual emotional features in order to transform them into audio signals with corresponding emotional aspects for users. Quantitative experiments show low mean absolute errors of 0.217 and 0.255 in the Lindsey and DEAP datasets, respectively, and similar global features in the spectrograms.is indicates that our model is able to appropriately perform domain transformation between visual and audio features. Based on experimental results, our model can e ectively generate an audio that matches the scene eliciting a similar emotion from the viewer in both datasets, and music generated by our model is also chosen more often (code available online at https://github.com/gcunhase/Emotional-Video-to-Audio-with-ANFIS-DeepRNN).Hinda i Ma hema ical P oblem in Enginee ing Vol me 2020, A icle ID 8478527, 15 age h ://doi.o g/

show abstract

Emotion recognition based on 3D fuzzy visual and EEG features in movie clips

Cited by 53 publications

References 26 publications

Wavelet-based study of valence–arousal model of emotions on EEG signals with LabVIEW

Wavelet-based study of valence–arousal model of emotions on EEG signals with LabVIEW

Review and Classification of Emotion Recognition Based on EEG Brain-Computer Interface System Research: A Systematic Review

Emotional Video to Audio Transformation Using Deep Recurrent Neural Networks and a Neuro-Fuzzy System

Contact Info

Product

Resources

About