A systematic review on affective computing: emotion models, databases, and recent advances

Wang, Yan; Song, Wei; Tao, Wei; Liotta, Antonio; Yang, Dawei; Li, Xinlei; Gao, Shuyong; Sun, Yixuan; Ge, Weifeng; Zhang, Wei; Zhang, Wenqiang

doi:10.1016/j.inffus.2022.03.009

Cited by 258 publications

(63 citation statements)

References 210 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To verify the flexibility and feasibility, the LSTM variants realized by the proposed system are applied to perform affective communication. Meanwhile, some existing state-of-the-art methods (including circuit-based and soft-based methods) [7,12,14,15] are also introduced for comparison purpose. Furthermore, four common performance metrics, i.e., the Accuracy, F1-Score, Recall, and time consumption are applied for objective evaluation.…”

Section: B Experimental Results and Analysismentioning

confidence: 99%

See 1 more Smart Citation

Design and Implementation of a Flexible Neuromorphic Computing System for Affective Communication via Memristive Circuits

Dong

Lai

et al. 2023

IEEE Commun. Mag.

View full text Add to dashboard Cite

Neuromorphic computing is expected to realize fast and energy-efficient artificial neural networks and address the inherent limitations of von Neumann architectures in dedicated communication applications. To realize this vision, we identify the existing challenges in neuromorphic computing and provide a specific solution from the perspectives of device, circuit, and system. At the device level, we fabricate a metal oxide-based memristor with high stability, low power, and good scalability, serving as the fundamental component of neuromorphic computing system. At the circuit level, the basic circuit units and necessary peripheral circuits are designed to realize efficient vector-matrix multiplication and different functions, including nonlinear activation operation, subtraction operation, added operation, etc. At the system level, a flexible neuromorphic computing system with a hardware-friendly training approach is proposed, it can perform affective communication with good trade-off between accuracy and time consumption. This study is expected to achieve the deep integration of nanotechnology, energy-efficient integrated circuits, and neuromorphic computing systems into communication applications.

show abstract

Section: B Experimental Results and Analysismentioning

confidence: 99%

“…Realization of affective communication indicates that computers could sense, recognize, and respond to human emotion, which is a great breakthrough in intelligent computing [12]. For validation, the proposed neuromorphic computing system is used to perform affective communication as present in this sub-section.…”

Section: Application In Affective Communicationmentioning

confidence: 99%

Design and Implementation of a Flexible Neuromorphic Computing System for Affective Communication via Memristive Circuits

Dong

Lai

et al. 2023

IEEE Commun. Mag.

View full text Add to dashboard Cite

show abstract

“…Automatic emotion recognition, as the first step to enable machines to have emotional intelligence, has been an active research area for the past two decades. Video emotion recognition (VER) refers to predicting the emotional states of the target person by analyzing information from different cues such as facial actions, acoustic characteristics and spoken language (Rouast et al, 2019 ; Wang et al, 2022 ). At the heart of this task is how to effectively learn emotional salient representations from multiple modalities including audio, visual, and text.…”

Section: Introductionmentioning

confidence: 99%

Multimodal interaction enhanced representation learning for video emotion recognition

2022

View full text Add to dashboard Cite

Video emotion recognition aims to infer human emotional states from the audio, visual, and text modalities. Previous approaches are centered around designing sophisticated fusion mechanisms, but usually ignore the fact that text contains global semantic information, while speech and face video show more fine-grained temporal dynamics of emotion. From the perspective of cognitive sciences, the process of emotion expression, either through facial expression or speech, is implicitly regulated by high-level semantics. Inspired by this fact, we propose a multimodal interaction enhanced representation learning framework for emotion recognition from face video, where a semantic enhancement module is first designed to guide the audio/visual encoder using the semantic information from text, then the multimodal bottleneck Transformer is adopted to further reinforce the audio and visual representations by modeling the cross-modal dynamic interactions between the two feature sequences. Experimental results on two benchmark emotion databases indicate the superiority of our proposed method. With the semantic enhanced audio and visual features, it outperforms the state-of-the-art models which fuse the features or decisions from the audio, visual and text modalities.

show abstract

“…Besides measuring users’ emotion with the Pleasure-Arousal-Dominance (PAD) Emotion Model, we take personality traits into account. Research has shown that users’ emotional states are also influenced by their personality traits ( Wang et al, 2022 ). Emotional states are temporary and have time dependencies.…”

Section: Introductionmentioning

confidence: 99%

Evaluating Users’ Emotional Experience in Mobile Libraries: An Emotional Model Based on the Pleasure-Arousal-Dominance Emotion Model and the Five Factor Model

et al. 2022

View full text Add to dashboard Cite

As a part of user experience, user emotion has rarely been studied in mobile libraries. Specifically, with the proposed emotional model in combination with the Pleasure-Arousal-Dominance (PAD) Emotion Model and the Five Factor Model (FFM), we evaluate user emotions on the mobile library’s three IS features (i.e., user interface, interaction quality, and service environment). An experience procedure with three tasks has been designed to collect data. 50 participants were enrolled, and they were asked to fill in questionnaires right after the experience. The correlations among the PAD emotions were examined. Specifically, users have a low perception of pleasure (P), high perception of arousal (A), and low perception of dominance (D). However, these three emotional states were not always significantly correlated with each other. This study extends mobile library research by focusing on users’ emotional experience. Specifically, the detailed PAD emotions have been examined. This study provides a new approach for application developers and managers to evaluate the user experience of an application.

show abstract

A systematic review on affective computing: emotion models, databases, and recent advances

Cited by 258 publications

References 210 publications

Design and Implementation of a Flexible Neuromorphic Computing System for Affective Communication via Memristive Circuits

Design and Implementation of a Flexible Neuromorphic Computing System for Affective Communication via Memristive Circuits

Multimodal interaction enhanced representation learning for video emotion recognition

Evaluating Users’ Emotional Experience in Mobile Libraries: An Emotional Model Based on the Pleasure-Arousal-Dominance Emotion Model and the Five Factor Model

Contact Info

Product

Resources

About