Evaluating facial recognition services as interaction technique for recommender systems

Abstract

Recommender systems are tools and techniques to assist users in the content selection process thereby coping with the problem of information overload. For recommender systems, user authentication and feedback gathering are of crucial importance. However, the typical user authentication with username / password and feedback method with a star rating system are not user friendly and often bypassed. This article proposes an alternative method for user authentication based on facial recognition and an automatic feedback gathering method by detecting various face characteristics such as emotions. We studied the use case of video watching. Photos made with the front-facing camera of a tablet, smartphone, or smart TV are used as input of a facial recognition service. The persons in front of the screen can be identified. During video watching, implicit feedback for the video content is automatically gathered through emotion recognition, attention measurements, and behavior analysis. An evaluation with a test panel showed that the recognized emotions are correlated with the user's star ratings and that happiness can be most accurately detected. So as the main contribution, this article indicates that emotion recognition might be used as an alternative feedback mechanism for recommender systems.

Affective video recommender systems: A survey

“…Facial expressions/features Soleymani and Pantic, 2012;Zhao et al, 2013;Boughrara et al, 2016;Choi et al, 2016;Kaklauskas et al, 2016;Mahata et al, 2017;Diaz et al, 2018;Fonnegra, 2018;Hewitt and Gunes, 2018;Kaklauskas et al, 2018;Bohlin et al, 2019;Soni et al, 2019;De Pessemier et al, 2020;Mishra et al, 2020;Leite et al, 2022 Skin-estimated pulse/heart rate Dabas et al, 2018;Diaz et al, 2018;Shu et al, 2018a;Bohlin et al, 2019;Soni et al, 2019;Ðor dević Čegar et al, 2020Mood Winoto and Tang, 2010EDA Ðor dević Čegar et al, 2020BA Alhagry, 2017Liu et al, 2017;Kwon et al, 2018;Ogawa et al, 2018;Yang et al, 2019;Ðor dević Čegar et al, 2020 User interactions Niu et al, 2013;Niu et al, 2016GSR Kwon et al, 2018 Body gestures Hassib et al, 2017 Perceived connotative properties Martha and Larson, 2013;Zhang and Zhang, 2017 Movie reviews/comments/web recordings Mulholland et al, 2017;Yenter, 2017;Tripathi et al, 2019;Krishnamurthy, 2020;Pan et al, 2020;Breitfuss et al...…”

“…Pearson's chi-square test and the dependent t-test Arapakis et al, 2009a Mean accuracy Zhao et al, 2011Zhao et al, , 2013Tkalčič et al, 2013b;Fan et al, 2016;Alhagry, 2017;Liu et al, 2017;Yenter, 2017;Zhang and Zhang, 2017;Dabas et al, 2018;Fonnegra, 2018;Hewitt and Gunes, 2018;Kwon et al, 2018;Shu et al, 2018a;Zhang et al, 2018;Bohlin et al, 2019;Soni et al, 2019;Yang et al, 2019;De Pessemier et al, 2020;Krishnamurthy, 2020;Mishra et al, 2020;Nie et al, 2020;Qi et al, 2021;Leite et al, 2022 Precision/recall/F1 Niu et al, 2013;Shi et al, 2013;Liu et al, 2017;Ogawa et al, 2018;Zhang et al, 2018;Tripathi et al, 2019;Yang et al, 2019;Krishnamurthy, 2020;Mishra et al, 2020;Cao et al, 2022 MAE Winoto andTang, 2010;Choi et al, 2016MSE/RMSE Boughrara et al, 2016Hewitt and Gunes, 2018;Tripathi et al, 2019;Zhu et al, 2019;Ðor dević Čegar et al, 2020ROC Winoto and Ta...…”

“…Dabas et al (2018) studied human emotions with the help of electroencephalogram (EEG) signals. De Pessemier et al (2019) facilitated HRI for users to watch videos by an automated procedure based on facial recognition. The automatic feedback is gathered when users play the videos using a front-facing camera.…”

Video Quality Prediction: An Exploratory Study With Valence and Arousal Signals

“…Facial expression recognition is a less intrusive technique that identifies emotions [6] by analyzing facial characteristics. This recognition process is often based on deep learning approaches trained on large datasets of photos or videos [7], [8].…”

“…By utilizing facial expression recognition for recognizing user emotion according to the discrete emotion theory [9], the work [8] shows that facial expression data correlates with explicit user feedback in a 10-star rating mechanism. The work [10] utilized facial expressions (in six classes) and skinestimated pulse (using PPG [11]) to predict a given user rating and whether a user would recommend a particular video.…”