Customer Gaze Estimation in Retail Using Deep Learning

Abstract

At present, intelligent computing applications are widely used in different domains, including retail stores. The analysis of customer behaviour has become crucial for the benefit of both customers and retailers. In this regard, the novel concept of remote gaze estimation using deep learning has shown promising results in analyzing customer behaviour in retail due to its scalability, robustness, low cost, and uninterrupted nature. This study presents a three-stage, three-attention-based deep convolutional neural network for remote gaze estimation in retail using only image data. In the first stage, we design a mechanism to estimate the 3D gaze of the subject using image data and monocular depth estimation. The second stage presents a novel three-attention mechanism to estimate the gaze in the wild from field-of-view, depth range, and object channel attentions. The third stage generates the gaze saliency heatmap from the output attention map of the second stage. We train and evaluate the proposed model on the benchmark GOO-Real dataset and compare the results with baseline models. Further, we adapt our model to real-retail environments by introducing a novel Retail Gaze dataset. Extensive experiments demonstrate that our approach significantly improves remote gaze target estimation performance on GOO-Real and Retail Gaze datasets. INDEX TERMS Computer vision, deep learning, gaze estimation, retail customer behaviour

Gaze Estimation Method Combining Facial Feature Extractor with Pyramid Squeeze Attention Mechanism

“…However, it suffered from slow real-time tracking due to its inability to locate facial feature coordinates. Senarath et al proposed a three-stage, three-attention deep convolutional neural network for retail remote gaze estimation using image data [29]. Luo et al proposed a collaborative network-based gaze estimation model with an attention mechanism that assigns appropriate weights between eye and facial features and achieved more accurate gaze estimation [30].…”

Smart and Sentient Retail High Streets

“…Deep learning is also employed for automation of customer analysis. Applications have included the use of deep learning to perform customer density mapping within stores [302], customer tracking in video [303], detection of customer gaze upon products on shelves [304,305], detection of shopper demographics and emotion recognition from facial expression [306], and even association of product detection with customer pose detection to determine customer purchase behavior [307]. The use of deep learning outdoors, on retail high streets, is less prevalent in the existing literature; however, a number of applications have been introduced.…”

A-DisETrac Advanced Analytic Dashboard for Distributed Eye Tracking

“…Despite wide adoption, these measures are geared towards single-user studies and are thus challenging to scale for multi-user studies. To elaborate, since eye trackers are designed to capture eye movements of one individual at a time, eye-tracking studies are often carried out as single-user experiments (Jayawardena et al 2021b;Mahanama et al 2022c;Michalek et al 2019;Senarath et al 2022) in isolated environments (Mahanama 2022a(Mahanama , 2021. Moreover, these measures only capture individual-level behaviors and do not account for inter-individual interactions.…”