GaTector: A Unified Framework for Gaze Object Prediction

Wang, Binglu; Hu, T.; Li, Baoshan; Chen, Xiaojuan; Zhang, Zhijie

doi:10.1109/cvpr52688.2022.01898

Cited by 30 publications

(15 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[37] dataset. Compared with GaTector [44], GTR achieves a 9.62 AP gain (+19.8%) for GO-D and 0.19 mAP gain (+43.5%) for joint GO-D and human head detection on GOO-Real subset. Moreover, GTR is designed as a single-stage and end-to-end pipeline while prior methods invariably require an additional object detector, which is a fine-tuned YOLO-V4 [1] by following standard protocols.…”

Section: Performance Comparison To State-of-the-art Methodsmentioning

confidence: 99%

“…It also propose a new Gaze On Objects (GOO) dataset that is composed of a large set of synthetic images (GOO-Synth) supplemented by a smaller subset of real images (GOO-Real) of people looking at objects in a retail environment. On this basis, Wang et al [44] propose a new method GaTector for gaze object detection, which leveraged an additional object detector (YOLOV4 [1]) to recognize target objects.…”

Section: Related Workmentioning

confidence: 99%

“…Existing detection systems frame gaze following detection as two separate tasks, i.e., detecting where each human is looking at (gaze location detection, GL-D) [6,11,38] and detecting what each human is looking at (gaze object detection, GO-D) [37,44]. Specifically, the former aims to predict the gaze location of each human (i.e., a heatmap, the magnitude of different locations indicates the possibility that the location is the gaze point), while the latter further identify the category of the object being looked at (i.e., a bounding box with object shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

End-to-End Human-Gaze-Target Detection with Transformers

Min

Duan

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

This paper proposes an efficient and effective method for joint gaze location detection (GL-D) and gaze object detection (GO-D), i.e., gaze following detection. Current approaches frame GL-D and GO-D as two separate tasks, employing a multi-stage framework where human head crops must first be detected and then be fed into a subsequent GL-D sub-network, which is further followed by an additional object detector for GO-D. In contrast, we reframe the gaze following detection task as detecting human head locations and their gaze followings simultaneously, aiming at jointly detect human gaze location and gaze object in a unified and single-stage pipeline. To this end, we propose GTR, short for Gaze following detection TRansformer, streamlining the gaze following detection pipeline by eliminating all additional components, leading to the first unified paradigm that unites GL-D and GO-D in a fully end-to-end manner. GTR enables an iterative interaction between holistic semantics and human head features through a hierarchical structure, inferring the relations of salient objects and human gaze from the global image context and resulting in an impressive accuracy. Concretely, GTR achieves a 12.1 mAP gain (25.1%) on GazeFollowing and a 18.2 mAP gain (43.3%) on VideoAttentionTarget for GL-D, as well as a 19 mAP improvement (45.2%) on GOO-Real for GO-D. Meanwhile, unlike existing systems detecting gaze following sequentially due to the need for a human head as input, GTR has the flexibility to comprehend any number of people's gaze followings simultaneously, resulting in high efficiency. Specifically, GTR introduces over a ×9 improvement in FPS and the relative gap becomes more pronounced as the human number grows.

show abstract

Section: Performance Comparison To State-of-the-art Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

End-to-End Human-Gaze-Target Detection with Transformers

Min

Duan

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…In particular, we consider the visual selective attention (VSA) towards the scene, which is the process of directing the gaze to relevant visual stimuli while ignoring the irrelevant ones in the environment [9]. The task of detecting the target being looked at by a person in an image or video is known as attention target detection [2,10,18,22,24,26,37,45]. However, in the scenario presented in this work we have a special case of this task in which the interest targets are always outside the video frame.…”

Section: Gaze Analysismentioning

confidence: 99%

“…However, in the scenario presented in this work we have a special case of this task in which the interest targets are always outside the video frame. While in most cases the problem is tackled in the 2D image space [10,37,45], we decided to extend our approach to work in a fully 3D manner. The scenario consists of 3 shop windows, where a camera has been placed inside each one to record the outside.…”

Section: Gaze Analysismentioning

confidence: 99%