An Affect-Based Multimodal Video Recommendation System

Kaklauskas, Artūras; Gudauskas, Renaldas; Kozlovas, Matas; Peciure, Lina; Lepkova, Natalija; Cerkauskas, Justas; Banaitis, Audrius

doi:10.24846/v25i1y201601

Cited by 8 publications

(2 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Facial expressions/features Soleymani and Pantic, 2012;Zhao et al, 2013;Boughrara et al, 2016;Choi et al, 2016;Kaklauskas et al, 2016;Mahata et al, 2017;Diaz et al, 2018;Fonnegra, 2018;Hewitt and Gunes, 2018;Kaklauskas et al, 2018;Bohlin et al, 2019;Soni et al, 2019;De Pessemier et al, 2020;Mishra et al, 2020;Leite et al, 2022 Skin-estimated pulse/heart rate Dabas et al, 2018;Diaz et al, 2018;Shu et al, 2018a;Bohlin et al, 2019;Soni et al, 2019;Ðor dević Čegar et al, 2020Mood Winoto and Tang, 2010EDA Ðor dević Čegar et al, 2020BA Alhagry, 2017Liu et al, 2017;Kwon et al, 2018;Ogawa et al, 2018;Yang et al, 2019;Ðor dević Čegar et al, 2020 User interactions Niu et al, 2013;Niu et al, 2016GSR Kwon et al, 2018 Body gestures Hassib et al, 2017 Perceived connotative properties Martha and Larson, 2013;Zhang and Zhang, 2017 Movie reviews/comments/web recordings Mulholland et al, 2017;Yenter, 2017;Tripathi et al, 2019;Krishnamurthy, 2020;Pan et al, 2020;Breitfuss et al...…”

Section: Audience Responses Publicationsmentioning

confidence: 99%

Affective video recommender systems: A survey

Wang

Zhao

2022

Front. Neurosci.

View full text Add to dashboard Cite

Traditional video recommendation provides the viewers with customized media content according to their historical records (e.g., ratings, reviews). However, such systems tend to generate terrible results if the data is insufficient, which leads to a cold-start problem. An affective video recommender system (AVRS) is a multidiscipline and multimodal human-robot interaction (HRI) system, and it incorporates physical, physiological, neuroscience, and computer science subjects and multimedia resources, including text, audio, and video. As a promising research domain, AVRS employs advanced affective analysis technologies in video resources; therefore, it can solve the cold-start problem. In AVRS, the viewers’ emotional responses can be obtained from various techniques, including physical signals (e.g., facial expression, gestures, and speech) and internal signals (e.g., physiological signals). The changes in these signals can be detected when the viewers face specific situations. The physiological signals are a response to central and autonomic nervous systems and are mostly involuntarily activated, which cannot be easily controlled. Therefore, it is suitable for reliable emotion analysis. The physical signals can be recorded by a webcam or recorder. In contrast, the physiological signals can be collected by various equipment, e.g., psychophysiological heart rate (HR) signals calculated by echocardiogram (ECG), electro-dermal activity (EDA), and brain activity (GA) from electroencephalography (EEG) signals, skin conductance response (SCR) by a galvanic skin response (GSR), and photoplethysmography (PPG) estimating users’ pulse. This survey aims to provide a comprehensive overview of the AVRS domain. To analyze the recent efforts in the field of affective video recommendation, we collected 92 relevant published articles from Google Scholar and summarized the articles and their key findings. In this survey, we feature these articles concerning AVRS from different perspectives, including various traditional recommendation algorithms and advanced deep learning-based algorithms, the commonly used affective video recommendation databases, audience response categories, and evaluation methods. Finally, we conclude the challenge of AVRS and provide the potential future research directions.

show abstract

Section: Audience Responses Publicationsmentioning

confidence: 99%

Affective video recommender systems: A survey

Wang

Zhao

2022

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…These automated systems can recognize and interpret the emotional states of a person through physical and physiological measures. Physical conditions represent communicative signals such as facial expressions [46,170,171], speech detection (speech) [47,172], body gestures [47,173], and eyetracking when viewing interactive content [32,174]. Whereas the physiological measurements involve the recording of bodily variations such as the change in temperature and the increase in blood pressure [1][2][3].…”

Section: Emotion Recognitionmentioning

confidence: 99%

Tourist Recommender Systems Based on Emotion Recognition—A Scientometric Review

2020

View full text Add to dashboard Cite

Recommendation systems have overcome the overload of irrelevant information by considering users’ preferences and emotional states in the fields of tourism, health, e-commerce, and entertainment. This article reviews the principal recommendation approach documents found in scientific databases (Elsevier’s Scopus and Clarivate Web of Science) through a scientometric analysis in ScientoPy. Research publications related to the recommenders of emotion-based tourism cover the last two decades. The review highlights the collection, processing, and feature extraction of data from sensors and wearables to detect emotions. The study proposes the thematic categories of recommendation systems, emotion recognition, wearable technology, and machine learning. This paper also presents the evolution, trend analysis, theoretical background, and algorithmic approaches used to implement recommenders. Finally, the discussion section provides guidelines for designing emotion-sensitive tourist recommenders.

show abstract