Real-time 3D gaze analysis in mobile applications

Abstract

This paper presents a system for real-time analysis of 3D gaze data arising in mobile applications. Our system allows users to freely move in a known 3D environment while their gaze is computed on arbitrarily shaped objects. The scanpath is analysed fully automatically using fixations and areas-of-interest - all in 3D and real time. Furthermore, the scanpath can be visualized in parallel in a 3D model of the environment. This enables to observe the scanning behaviour of a subject. We describe how this has been realized for a commercial off-the-shelf mobile eye tracker utilizing an inside-out tracking mechanism for head pose estimation. Moreover, we show examples of real gaze data collected in a museum

Model-based real-time visualization of realistic three-dimensional heat maps for mobile eye tracking and eye tracking in virtual reality

“…In previous work in the European project ARtSENSE, Hammer et al [2013] used a 3D model of the environment and outside-in tracking based on the scene-camera video of the eye-tracking system to determine 3D-fixation locations during the viewer's visit to a museum. Based on this work, they developed a GPU-accelerated visualization-system for heat maps [2014].…”

“…From this perspective, the algorithm computes a so-called shadow map (see Section 5.1) to determine partial occlusions up to the depth of the viewing event's point of regard. This procedure is similar to the approach used by Hammer et al [2013], however, the technical implementation is different, as will be explained in Section 5.1.…”

EyeSee3D 2.0

“…In the frame of the ARtSENSE Projekt, Hammer et al [Hammer et al 2013] used a Dikablis Wireless monocular eye-tracking system as an interaction device for a museum assistant. They reconstructed depth information of the point of regard based on the 2D marker-tracking capabilities of the Diablis software and cast a ray into their static 3D situation model.…”

“…Third, on the situation-model level, an abstract representation of the current interaction context, called situation model, is used. Such model-based approaches try to reconstruct the position and orientation of the eye-tracking device and use a 3D model representation of the environment to determine the object of interest [Hammer et al 2013;Pfeiffer and Renner 2014].…”

Realistic heatmap visualization for interactive analysis of 3D gaze data

“…in advertisement departments for analyzing the effectiveness of adverts in print media or television. But with modern, fast and accurate mobile eye tracking systems, even more complicated, three-dimensional scenarios like in museums [Hammer et al 2013], shops [Harwood et al 2013] or even in sports [Campbell et al 2013] can be analyzed.…”