GOO: A Dataset for Gaze Object Prediction in Retail Environments

Abstract

One of the most fundamental and information-laden actions humans do is to look at objects. However, a survey of current works reveals that existing gaze-related datasets annotate only the pixel being looked at, and not the boundaries of a specific object of interest. This lack of object annotation presents an opportunity for further advancing gaze estimation research. To this end, we present a challenging new task called gaze object prediction, where the goal is to predict a bounding box for a person's gazed-at object. To train and evaluate gaze networks on this task, we present the Gaze On Objects (GOO) dataset. GOO is composed of a large set of synthetic images (GOO-Synth) supplemented by a smaller subset of real images (GOO-Real) of people looking at objects in a retail environment. Our work establishes extensive baselines on GOO by re-implementing and evaluating selected state-of-the-art models on the task of gaze following and domain adaptation. Code is available 1 on github.

Customer Gaze Estimation in Retail Using Deep Learning

“…First, the hypothetical gaze distribution model, as defined in our previous work [12], represents the concept of the object channel. The model has surpassed the existing benchmark Area Under the Curve (AUC) and Angular error baselines on the GOO dataset [8], showing the importance of the object channel. Second, the Face3D model represents the depth channel and the novel concept of remote gaze estimation in 3D vector space.…”

“…Dataset GTE GF GOP GE in Retail Approach\Technique Bermejo et al [1] UcoHead, Own dataset ✓ --✓ CNN (Coarse-to-Fine) Recasense et al [11] Gaze Follow ✓ ✓ --CNN with shifted grids Tomas et al [8] GOO ✓ ✓ ✓ ✓ Existing CNN models Kellnhofer et al [17] Gaze360 ✓ --✓ CNN-LSTM Fang et al [13] Gaze360, Gaze Follow, VideoAttentionTarget ✓ ---Attention-based CNN Chong et al [23] Gaze Follow, VideoCoAtt, VideoAttentionTarget ✓ ✓ --CNN-LSTM Lian et al [24] Gaze Follow ✓ ✓ --Static-CNN Kodama et al [22] Own dataset ✓ ✓ --Static-CNN Khamis et al [25] Own…”

“…Then, in order to create the dual attention map, we aggregated the FoV attention map and depth attention map as given in (7). After that, to enhanced the saliency estimation we aggregated this dual attention map with hypothetical gaze distribution G (object channel) to create hypo dual attention map as in (8), where ⊗ denotes the element wise product.…”

“…However, the existing solutions only captures coarse touch-points of a shopper's journey and vulnerable to unconstrained environment settings. With the adaptation of computer vision technologies in gaze estimation, there has been eye tracking-based solutions for customer behaviour analysis in retail as well [1], [8]. Moreover, there are solutions based on virtual reality devices and headmounted displays, wearable eye tracker based solutions [9], and non-intrusive 3D eye tracking solutions [10].…”

“…The concept of Gaze Following, which was introduced by Recasense et al [11], refers to the identification of the object being looked at by a person, given the scene image. This concept has been extended by Tomas et al [8] and presented the idea of Gaze Object Prediction in retail, that refers to the task of predicting the bounding box for a human's gazed-at object. Both these concepts only require gaze estimation from the scene image, and it avoids the need to wear special types of devices to capture the eye gaze and remove the restrictions of manual calibration.…”