Classification of affects using head movement, skin color features and physiological signals

Monkaresi, Hamed; Hussain, M. Sazzad; Calvo, Rafael A.

doi:10.1109/icsmc.2012.6378149

Cited by 16 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The best result for unimodal fusion was obtained using feature-level fusion and also there is an improvement in the processing time compared to existing methods. The authors (Monkaresi, Hussain, & Calvo, 2012) have combined the head movement, physiology, and facial color using early fusion for the affect classification. It has achieved statistically higher accuracy compared to the unimodal approaches.…”

Section: Feature Level or Early Fusionmentioning

confidence: 99%

Multimodal sentimental analysis for social media applications: A comprehensive review

Chandrasekaran

Nguyen

Hemanth

2021

WIREs Data Min & Knowl

View full text Add to dashboard Cite

The analysis of sentiments is essential in identifying and classifying opinions regarding a source material that is, a product or service. The analysis of these sentiments finds a variety of applications like product reviews, opinion polls, movie reviews on YouTube, news video analysis, and health care applications including stress and depression analysis. The traditional approach of sentiment analysis which is based on text involves the collection of large textual data and different algorithms to extract the sentiment information from it. But multimodal sentimental analysis provides methods to carry out opinion analysis based on the combination of video, audio, and text which goes a way beyond the conventional text-based sentimental analysis in understanding human behaviors. The remarkable increase in the use of social media provides a large collection of multimodal data that reflects the user's sentiment on certain aspects. This multimodal sentimental analysis approach helps in classifying the polarity (positive, negative, and neutral) of the individual sentiments. Our work aims to present a survey of recent developments in analyzing the multimodal sentiments (involving text, audio, and video/image) which involve humanmachine interaction and challenges involved in analyzing them. A detailed survey on sentimental dataset, feature extraction algorithms, data fusion methods, and efficiency of different classification techniques are presented in this work.

show abstract

Section: Feature Level or Early Fusionmentioning

confidence: 99%

Multimodal sentimental analysis for social media applications: A comprehensive review

Chandrasekaran

Nguyen

Hemanth

2021

WIREs Data Min & Knowl

View full text Add to dashboard Cite

show abstract

“…The statistical data is stored according to objects color, eye color, eye movement, the position of eyes on the head, head sideways. Various recorded video files can also be analyzed using MATLAB and OpenCV [11]. In this article various geometric and chromatic features were used as image-based features.…”

Section: Literature Reviewmentioning

confidence: 99%

A Human-Robot Interaction System Calculating Visual Focus of Human’s Attention Level

Chakraborty¹,

Ahmed²,

Yousuf³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Attention is the mental awareness of human on a particular object or a piece of information. The level of attention indicates how intense the focus is on an object or an instance. In this study, several types of human attention level have been observed. After introducing image segmentation and detection technique for facial features, eyeball movement and gaze estimation were measured. Eye movement were assessed using the video data, and a total of 10197 data instances were manually labelled for the attention level. Then Artificial Neural Network (ANN) and Recurrent Neural Network-Long Short Term Memory (LSTM) based Deep learning (DL) architectures have been proposed for analysing the data. Next, the trained DL model has been implanted into a robotic system that is capable of detecting various features; ultimately leading to the calculation of visual attention for reading, browsing, and writing purposes. This system is capable of checking the attention level of the participants and also can detect if participants are present or not. Based on a certain level of visual focus of attention (VFOA), this system interacts with the person, generates awareness and establishes verbal or visual communication with that person. The proposed ML techniques have achieved almost 99.24% validation accuracy and 99.43% test accuracy. It is also shown in the comparative study that, since the dataset volumes are limited, ANN is more suitable for attention level calculation than RNN-LSTM. We hope that the implemented robotic structure manifests the real-world implication of the proposed method.

show abstract

“…The presentation method is standardized such that all individuals have the same viewing experience. For example, the images could each be presented for ten seconds on a computer screen that is a fixed distance from where the individual is sitting, with a constant screen resolution, screen brightness, and image size (Monkaresi, Hussain, & Calvo, 2012). The images themselves can be selected on the basis of which emotions should be elicited from a database of standardized images.…”

Section: Emotional Imagesmentioning

confidence: 99%

“…They used this data to train unimodal and multimodal emotion detectors using EEG and peripheral physiological signals. Leon et al (2007) and Monkaresi et al (2012) followed similar protocols, but collected self-reports after every IAPS image, instead of after sets of images.…”

Section: Emotional Imagesmentioning

confidence: 99%

“…Naturalistic data, on the other hand, is certainly ecologically valid, but notoriously difficult to collect. The middle road, in which specific emotions are elicited, may be a good compromise for AC researchers and has been used to great effect already (see, e.g., Bailenson et al, 2008;Khalili & Moradi, 2009;Koelstra et al, 2011;Leon, Clarke, Callaghan, and Sepulveda, 2007;Monkaresi, Hussain, Calvo, 2012;Soleymani, Pantic, & Pun, 2012). This chapter focuses on method to elicit particular affective states with an emphasis on methods that have been previously used or have considerable potential for use in AC research.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Affect Elicitation for Affective Computing

Kory

D’Mello

2014

Oxford Handbooks Online

View full text Add to dashboard Cite

The ability to reliably and ethically elicit affective states in the laboratory is critical when studying and developing systems that can detect, interpret, and adapt to human affect. Many methods for eliciting emotions have been developed. In general, they involve presenting a stimulus to evoke a response from one or more emotion response systems. The nature of the stimulus varies widely. Passive methods include the presentation of images, film clips, and music. Active methods can involve social or dyadic interactions with other people, or behavior manipulations in which an individual is instructed to adopt facial expressions, postures, or other emotionally-relevant behaviors. This chapter discusses exemplar methods of each type, discusses advantages and disadvantages of each method, and briefly summarizes some additional methods.Keywords/keyphrases affect elicitation, emotional images, emotional film clips, emotional music, backward masking, behavior manipulation, social interaction, dyadic interaction Glossary Affect elicitation: Methods used to evoke (or induce) affective responses in individuals. These methods generally involve presenting a stimulus or immersing the subject in a situation to evoke a response from one or more emotion response systems.The nature of the stimulus varies and could include the presentation of images, film, or music; adopting facial expressions or postures; and social or dyadic interactions, among others.Emotional images: Digital images or photographs that have been carefully selected and evaluated for their potential to evoke affective states in the viewer.Emotional film clips: Short movie segments, usually including both images and sound, which have been selected and evaluated for their potential to evoke affective states in the viewer.Emotional music: A recorded musical piece that has been selected and evaluated for its ability to evoke affective states in the listener.Backward masking: A method used to block conscious awareness of a visual stimulus. The target stimulus is shown to an individual very briefly (such as 15-60ms), followed immediately by a "mask" stimulus shown for a longer time (such as 500ms).Individuals report being consciously aware of only the mask.Behavior manipulation: A method in which individuals are instructed to adopt particular behaviorssuch as body postures or facial expressionsin order to induce particular affective states.Social interaction: A relationship between two or more individuals, that may be fleeting or enduring, in which an individual's actions and behavior are responsive to the actions and behavior of the other individuals. In one affect elicitation method, researchers try to create realistic social interaction scenarios that might evoke emotions in a more naturalistic context. Dyadic interaction: A social interaction specifically between two individuals (seeSocial Interaction). One affect elicitation method focuses on bringing pairs of individuals together to engage in an unrehearsed, minimally structured conversation in order to evoke aff...

show abstract

Classification of affects using head movement, skin color features and physiological signals

Cited by 16 publications

References 21 publications

Multimodal sentimental analysis for social media applications: A comprehensive review

Multimodal sentimental analysis for social media applications: A comprehensive review

A Human-Robot Interaction System Calculating Visual Focus of Human’s Attention Level

Affect Elicitation for Affective Computing

Contact Info

Product

Resources

About