The Emotracker: Visualizing Contents, Gaze and Emotions at a Glance

Abstract

In the last years, Affective Computing investigations have focused the efforts in the automatic extraction of human emotions and in increasing the success rates in the emotion recognition task. However, there is a lack of automatic tools that intuitively visualize the users' emotional information. In this work, we propose Emotracker, a novel tool based on the combination of eye tracking and facial emotional recognition technologies that allows the visualization of contents, user emotions and gaze at a glance, and offers a wide range of visualization options, including emotional saccade maps and emotional heat maps. The simultaneous consideration of gaze and emotion opens the door to the evaluation of user engagement, which is of great importance for content creators, but has also other promising fields of application in psychology research and, more specifically, in autism.

A Review on Automatic Facial Expression Recognition Systems Assisted by Multimodal Sensor Data

“…There are a handful of works which have focused on the combination of the eye tracker and other sensors’ input to recognize facial expressions. Emotracker is an application which tries to detect the emotional statues of individuals [62], which comprises two types ofsoftware (Tobii Studio [63] and Noldus Face Reader [64]). These tools keep robustness and accuracy even in real-world environments.…”

“…Module 1: We use data extracted from detailed-face sensors to improve the performance of the FER system and test the system using real-world background changes, for instance, from indoor to outdoor situations. As an eye tracker can reduce a large amount of face image processing [62], we can use it to efficiently do a coarse detection and thus, image/video processing for detecting and tracking face from other wild background environment is only required to refine the results when confidence is low. This module can improve the efficiency of the proposed method.…”