Towards an Affective Video Recommendation System

Diaz, Yancarlos; Alm, Cecilia Ovesdotter; Nwogu, Ifeoma; Bailey, Reynold

doi:10.1109/percomw.2018.8480130

Cited by 14 publications

(17 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We compared our method with other affective analysis based video recommendation approaches which use facial expressions such as [78], [79]. Our hybrid algorithms generally outperformed these models, by having a much lower RMSE against the MAE results for different scenarios as shown in [78] and better computed emotions as shown in [79]. The results are quite apparent as we monitor and learn real-time contextual user feedback that too with multidimensional user behavior to recommend content based on the affective alignment of the user with the watched video and users' history pattern.…”

Section: Discussionmentioning

confidence: 99%

EmoWare: A Context-Aware Framework for Personalized Video Recommendation Using Affective Video Sequences

Tripathi¹,

Ashwin²,

Guddeti³

2019

IEEE Access

View full text Add to dashboard Cite

With the exponential growth in areas of machine intelligence, the world has witnessed promising solutions to the personalized content recommendation. The ability of interactive learning agents to make optimal decisions in dynamic environments has been proven and very well conceptualized by reinforcement learning (RL). The learning characteristics of deep-bidirectional recurrent neural networks (DBRNN) in both positive and negative time directions has shown exceptional performance as generative models to generate sequential data in supervised learning tasks. In this paper, we harness the potential of the said two techniques and propose EmoWare (emotion-aware), a personalized, emotionally intelligent video recommendation engine, employing a novel context-aware collaborative filtering approach, where the intensity of users' spontaneous non-verbal emotional response toward the recommended video is captured through interactions and facial expressions analysis for decision-making and video corpus evolution with real-time feedback streams. To account for users' multidimensional nature in the formulation of optimal policies, RL-scenarios are enrolled using on-policy (SARSA) and off-policy (Q-learning) temporal-difference learning techniques, which are used to train DBRNN to learn contextual patterns and to generate new video sequences for the recommendation. System evaluation for a month with real users shows that the EmoWare outperforms the state-of-the-art methods and models users' emotional preferences very well with stable convergence. INDEX TERMS Reinforcement learning, Q-learning, SARSA, deep bidirectoinal recurrent neural network, multi-armed bandit, video recommendation, affectiva, intensities of emotions, emotion-based information retrieval.

show abstract

Section: Discussionmentioning

confidence: 99%

EmoWare: A Context-Aware Framework for Personalized Video Recommendation Using Affective Video Sequences

Tripathi¹,

Ashwin²,

Guddeti³

2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Therefore, an AVRS is a more complex domain in recommender systems. Several researchers tend to solve the AVRS problem by various methods, traditional models, or algorithms, including support vector machine/support vector regression (SVM/SVR) (Arapakis et al, 2009a), clustering (Song and Yang, 2022), AdaBoost (Zhao et al, 2013), matrix-based algorithm (MA) (Kaklauskas et al, 2018), collaborative filtering (CF) (Diaz et al, 2018), content-based filtering (CBF) (Deldjoo et al, 2018), knowledge graph (KG) (Breitfuss et al, 2021), genetic algorithms (GA) , hybrid recommendation systems (HRS) (Wakil et al, 2015), the combination of several traditional recommendation algorithms, etc. Deep learning (DL) has gradually penetrated the field of affective computing and promoted the development of video recommendations.…”

Section: The State-of-the-art Affective Video Recommendation Algorith...mentioning

confidence: 99%

“…Traditional methods SVM/SVR Arapakis et al, 2009a,b;Soleymani and Pantic, 2012;Srivastava and Roy, 2014;Sivakumar et al, 2015;Niu et al, 2017;Dabas et al, 2018;Bohlin et al, 2019Clustering Niu et al, 2013Niu et al, 2016AdaBoost Zhao et al, 2013Shu et al, 2018aMA Tkalčič et al, 2013bDnodxvndv et al, 2018;Kaklauskas et al, 2018CF Soleymani et al, 2009Winoto and Tang, 2010;Tkalčič et al, 2013b;Choi et al, 2016;Diaz et al, 2018CBF Shi et al, 2013Tkalčič et al, 2013b;Deldjoo et al, 2018KG Breitfuss et al, 2021Qi et al, 2021GA Yadati et al, 2014HRS Mugellini et al, 2014Wakil et al, 2015 Deep learning-based methods RL Tripathi et al, 2018;Leite et al, 2022 CNN Hewitt andGunes, 2018;Kwon et al, 2018;Yang et al, 2019;Zhu et al, 2019LSTM Alhagry, 2017Zhang and Zhang, 2017;Ogawa et al, 2018;Nie et al, 2020;Cao et al, 2022MLP Boughrara et al, 2016Ðor dević Čegar et al, 2020;Krishnamurthy, 2020…”

Section: Categories Algorithm/model Publicationsmentioning

confidence: 99%

“…Through this method, they provided accurate, personalized recommendations for new and existing users, thus solving the users' cold-start problem. In Diaz et al (2018), they designed a recommender entirely based on the impression data of viewers. When a user views a video, the recommender system retrieves the metric data from user information.…”

Section: Collaborative Filtering (Cf)mentioning

confidence: 99%

See 1 more Smart Citation

Affective video recommender systems: A survey

Wang

Zhao

2022

Front. Neurosci.

View full text Add to dashboard Cite

Traditional video recommendation provides the viewers with customized media content according to their historical records (e.g., ratings, reviews). However, such systems tend to generate terrible results if the data is insufficient, which leads to a cold-start problem. An affective video recommender system (AVRS) is a multidiscipline and multimodal human-robot interaction (HRI) system, and it incorporates physical, physiological, neuroscience, and computer science subjects and multimedia resources, including text, audio, and video. As a promising research domain, AVRS employs advanced affective analysis technologies in video resources; therefore, it can solve the cold-start problem. In AVRS, the viewers’ emotional responses can be obtained from various techniques, including physical signals (e.g., facial expression, gestures, and speech) and internal signals (e.g., physiological signals). The changes in these signals can be detected when the viewers face specific situations. The physiological signals are a response to central and autonomic nervous systems and are mostly involuntarily activated, which cannot be easily controlled. Therefore, it is suitable for reliable emotion analysis. The physical signals can be recorded by a webcam or recorder. In contrast, the physiological signals can be collected by various equipment, e.g., psychophysiological heart rate (HR) signals calculated by echocardiogram (ECG), electro-dermal activity (EDA), and brain activity (GA) from electroencephalography (EEG) signals, skin conductance response (SCR) by a galvanic skin response (GSR), and photoplethysmography (PPG) estimating users’ pulse. This survey aims to provide a comprehensive overview of the AVRS domain. To analyze the recent efforts in the field of affective video recommendation, we collected 92 relevant published articles from Google Scholar and summarized the articles and their key findings. In this survey, we feature these articles concerning AVRS from different perspectives, including various traditional recommendation algorithms and advanced deep learning-based algorithms, the commonly used affective video recommendation databases, audience response categories, and evaluation methods. Finally, we conclude the challenge of AVRS and provide the potential future research directions.

show abstract

“…Esses fenômenos afetivos aliados à emoção, são definidos por Scherer et al (2000), como sendo estados afetivos. A identificação e o entendimento de como esses estados afetam as relações têm sido alvo de estudos em áreas como a saúde Revista Novas Tecnologias na Educação ____________________________________________________________________________________________ V. 18 Nº 1, julho, 2020_______________________________________________________RENOTE DOI: (Prabhu et al, 2019), entretenimento (Diaz et al, 2018) e educação (Troussas e Virvou, 2020).…”

Section: Introductionunclassified

Sensibilidade ao contexto na identificação de estados afetivos aplicados à educação: um mapeamento sistemático

Dorneles

Barbosa

2020

RENOTE

View full text Add to dashboard Cite

Estudos apontam como tendência o uso de lentes emocionais e a relevância das emoções ou estados afetivos para as interações educacionais. Aliado a isso, e considerando as potencialidades da computação ubíqua e afetiva, cresce o entendimento da importância do uso de informações contextuais na identificação de sinais afetivos e sociais. Dessa forma, esse estudo apresenta uma revisão da literatura sobre o uso de informações de contextos na identificação de estados afetivos em ambientes educacionais. Os resultados indicam que os trabalhos fazem uso de canais visuais (50%) e fisiológicos (50%) associados principalmente a informações de contexto, relacionadas a atividades e identidade na detecção de estados afetivos. A revisão identificou ainda, o uso de 27 diferentes estados afetivos com predominância na valência das emoções positivas como felicidade e satisfação ou negativas como frustração e decepção.

show abstract

Towards an Affective Video Recommendation System

Cited by 14 publications

References 12 publications

EmoWare: A Context-Aware Framework for Personalized Video Recommendation Using Affective Video Sequences

EmoWare: A Context-Aware Framework for Personalized Video Recommendation Using Affective Video Sequences

Affective video recommender systems: A survey

Sensibilidade ao contexto na identificação de estados afetivos aplicados à educação: um mapeamento sistemático

Contact Info

Product

Resources

About