A Survey on Automatic Multimodal Emotion Recognition in the Wild

Sharma, Garima; Dhall, Abhinav

doi:10.1007/978-3-030-51870-7_3

Cited by 40 publications

(13 citation statements)

References 131 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Human-Computer Interaction (HCI) is an essential part of Artificial Intelligence (AI) research. Real-life HCI applications have been facilitated by research on automatic emotion recognition [8], thus improving quality of service and quality of life [9]. The speech signal is one of the key ways for humans to communicate, since it contains a substantial amount of paralinguistic information (i. e., emotion states, attitudes, etc.)…”

Section: Speech Emotion Recognitionmentioning

confidence: 99%

Robust Federated Learning Against Adversarial Attacks for Speech Emotion Recognition

Chen¹,

Sofiane²,

Zhao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Due to the development of machine learning and speech processing, speech emotion recognition has been a popular research topic in recent years. However, the speech data cannot be protected when it is uploaded and processed on servers in the internet-of-things applications of speech emotion recognition. Furthermore, deep neural networks have proven to be vulnerable to human-indistinguishable adversarial perturbations. The adversarial attacks generated from the perturbations may result in deep neural networks wrongly predicting the emotional states. We propose a novel federated adversarial learning framework for protecting both data and deep neural networks. The proposed framework consists of i) federated learning for data privacy, and ii) adversarial training at the training stage and randomisation at the testing stage for model robustness. The experiments show that our proposed framework can effectively protect the speech data locally and improve the model robustness against a series of adversarial attacks.

show abstract

Section: Speech Emotion Recognitionmentioning

confidence: 99%

Robust Federated Learning Against Adversarial Attacks for Speech Emotion Recognition

Chen¹,

Sofiane²,

Zhao³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Studies by Albert Mehrabian in the 80s (Mehrabian 1981) established the 7-38-55% rule, also known as the "3V rule": 7% of communication is verbal, 38% of communication is vocal and 55% of communication is visual. Multimodal Emotion Recognition approaches rely on a combination of facial, body and verbal signals to infer the emotion of a subject (Sharma and Dhall 2021). One state of the art example of multimodal emotion recognition is End-to-End Multimodal Emotion Recognition Using Deep Neural Networks (Tzirakis et al 2017), in which the network comprises of two parts: the multimodal feature extraction part and the RNN part.…”

Section: Multimodal Emotion Recognitionmentioning

confidence: 99%

When AI meets store layout design: a review

Nguyen

Martin

et al. 2022

Artif Intell Rev

View full text Add to dashboard Cite

An efficient store layout presents merchandise to attract customer attention and encourages customers to walk down more aisles which exposes them to more merchandise, which has been shown to be positively correlated with the sales. It is one of the most effective in-store marketing tactics which can directly influence customer decisions to boost store sales and profitability. The recent development of Artificial Intelligence techniques, especially with its sub-fields in Computer Vision and Deep Learning, has enabled retail stores to take advantage of existing CCTV infrastructure to extract in-store customer and business insights. This research aims to conduct a comprehensive review on existing approaches in store layout design and modern AI techniques that can be utilized in the layout design task. Based on this review, we propose an AI-powered store layout design framework. This framework applies advanced AI and data analysis techniques on top of existing CCTV video surveillance infrastructure to understand, predict and suggest a better store layout.

show abstract

“…Numerous publications in this field are collected through extensive surveys, including [23][24][25]. Since the audio channel does not provide continuity in recognising emotions, and the effectiveness of recognition depends on many factors, including voice quality, this source of information is very often used in multimodal systems [26][27][28]. For example, an interesting emotographic model analyses indicators residing in standard multimodal data produced by commonly used applications and Internet of things (IoT) devices to interpret human emotional state [22].…”

Section: Emotion Recognitionmentioning

confidence: 99%

A Review of Emotion Recognition Methods Based on Data Acquired via Smartphone Sensors

Kołakowska

Szwoch

2020

Sensors

View full text Add to dashboard Cite

In recent years, emotion recognition algorithms have achieved high efficiency, allowing the development of various affective and affect-aware applications. This advancement has taken place mainly in the environment of personal computers offering the appropriate hardware and sufficient power to process complex data from video, audio, and other channels. However, the increase in computing and communication capabilities of smartphones, the variety of their built-in sensors, as well as the availability of cloud computing services have made them an environment in which the task of recognising emotions can be performed at least as effectively. This is possible and particularly important due to the fact that smartphones and other mobile devices have become the main computer devices used by most people. This article provides a systematic overview of publications from the last 10 years related to emotion recognition methods using smartphone sensors. The characteristics of the most important sensors in this respect are presented, and the methods applied to extract informative features on the basis of data read from these input channels. Then, various machine learning approaches implemented to recognise emotional states are described.

show abstract

A Survey on Automatic Multimodal Emotion Recognition in the Wild

Cited by 40 publications

References 131 publications

Robust Federated Learning Against Adversarial Attacks for Speech Emotion Recognition

Robust Federated Learning Against Adversarial Attacks for Speech Emotion Recognition

When AI meets store layout design: a review

A Review of Emotion Recognition Methods Based on Data Acquired via Smartphone Sensors

Contact Info

Product

Resources

About