Multiple-Gaze Geometry: Inferring Novel 3D Locations from Gazes Observed in Monocular Video

Brau, Ernesto; Guan, Jinyan; Jeffries, Tanya; Barnard, Kobus

doi:10.1007/978-3-030-01225-0_38

Cited by 19 publications

(15 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For learning-based approaches, we report the mean and standard deviation over five runs. Results on the [25] distant time frame is important, and this is difficult to achieve with RNN (or LSTM) processing data sequentially [37].…”

Section: Methodsmentioning

confidence: 99%

“…This could be obtained by gathering a dataset of real-life scenarios which could be use either as training data or to improve the quality of the generative model. The only methods from the literature that we are aware of are [24] and [25]. In both cases, neither the data nor the code have been made available online.…”

Section: Methodsmentioning

confidence: 99%

“…However, it fails when some people are undetected and the group structures are wrongly estimated, or when a person is isolated and should not be integrated into a group structure. By contrast, both [24] and [25] independently propose to use 3D intersection of gazes in a probabilistic framework to estimate locations of objects of interest, possibly outside the camera field of view. The methods achieve good levels of performance -even though [24] lacks quantitative evaluation.…”

Section: Related Workmentioning

confidence: 99%

“…lem, without the restriction of being limited to the image plane. Only very few works exist in this direction [23], [24], [25], and all employ strong social or geometric assumptions.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Extended Gaze Following: Detecting Objects in Videos Beyond the Camera Field of View

Mâsse¹,

Lathuilière²,

Mesejo³

et al. 2019

2019 14th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2019)

View full text Add to dashboard Cite

In this paper we address the problems of detecting objects of interest in a video and of estimating their locations, solely from the gaze directions of people present in the video. Objects can be indistinctly located inside or outside the camera field of view. We refer to this problem as extended gaze following. The contributions of the paper are the followings. First, we propose a novel spatial representation of the gaze directions adopting a top-view perspective. Second, we develop several convolutional encoder/decoder networks to predict object locations and compare them with heuristics and with classical learning-based approaches. Third, in order to train the proposed models, we generate a very large number of synthetic scenarios employing a probabilistic formulation. Finally, our methodology is empirically validated using a publicly available dataset.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…lem, without the restriction of being limited to the image plane. Only very few works exist in this direction [23], [24], [25], and all employ strong social or geometric assumptions.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Extended Gaze Following: Detecting Objects in Videos Beyond the Camera Field of View

Mâsse¹,

Lathuilière²,

Mesejo³

et al. 2019

2019 14th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2019)

View full text Add to dashboard Cite

show abstract

“…Joint attention is also the central topic in [ 100 ], where a Bayesian generative statistical model for temporal scene understanding using probabilistic graphical modeling notation is introduced. The model captures the joint probability of camera parameters, locations of people, their gaze, what they are looking at, and locations of visual attention.…”

Section: Gaze Tracking By Scene Analysismentioning

confidence: 99%

When I Look into Your Eyes: A Survey on Computer Vision Contributions for Human Gaze Estimation and Tracking

Cazzato

Leo

Distante

et al. 2020

Sensors

View full text Add to dashboard Cite

The automatic detection of eye positions, their temporal consistency, and their mapping into a line of sight in the real world (to find where a person is looking at) is reported in the scientific literature as gaze tracking. This has become a very hot topic in the field of computer vision during the last decades, with a surprising and continuously growing number of application fields. A very long journey has been made from the first pioneering works, and this continuous search for more accurate solutions process has been further boosted in the last decade when deep neural networks have revolutionized the whole machine learning area, and gaze tracking as well. In this arena, it is being increasingly useful to find guidance through survey/review articles collecting most relevant works and putting clear pros and cons of existing techniques, also by introducing a precise taxonomy. This kind of manuscripts allows researchers and technicians to choose the better way to move towards their application or scientific goals. In the literature, there exist holistic and specifically technological survey documents (even if not updated), but, unfortunately, there is not an overview discussing how the great advancements in computer vision have impacted gaze tracking. Thus, this work represents an attempt to fill this gap, also introducing a wider point of view that brings to a new taxonomy (extending the consolidated ones) by considering gaze tracking as a more exhaustive task that aims at estimating gaze target from different perspectives: from the eye of the beholder (first-person view), from an external camera framing the beholder’s, from a third-person view looking at the scene where the beholder is placed in, and from an external view independent from the beholder.

show abstract

LAEO-Net: Revisiting People Looking at Each Other in Videos

Marín-Jiménez¹,

Kalogeiton

Medina-Suarez

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Figure 1: Intimacy or hostility? Head pose, along with body pose and facial expressions, is a rich source of information for interpreting human interactions. Being able to automatically understand the non-verbal cues provided by the relative head orientations of people in a scene enables a new level of human-centric video understanding. Green and red/orange heads represent LAEO and non-LAEO cases, respectively. Video source of second row: https://youtu.be/B3eFZMvNS1UAbstract Capturing the 'mutual gaze' of people is essential for understanding and interpreting the social interactions between them. To this end, this paper addresses the problem of detecting people Looking At Each Other (LAEO) in video sequences. For this purpose, we propose LAEO-Net, a new deep CNN for determining LAEO in videos. In contrast to previous works, LAEO-Net takes spatio-temporal tracks as input and reasons about the whole track. It consists of three branches, one for each character's tracked head and one for their relative position. Moreover, we introduce two new LAEO datasets: UCO-LAEO and AVA-LAEO. A thorough experimental evaluation demonstrates the ability of LAEO-Net to successfully determine if two people are LAEO and the temporal window where it happens. Our model achieves state-of-the-art results on the existing TVHID-LAEO video dataset, significantly outperforming previous approaches. Finally, we apply LAEO-Net to social network analysis, where we automatically infer the social relationship between pairs of people based on the frequency and duration that they LAEO.

show abstract

Multiple-Gaze Geometry: Inferring Novel 3D Locations from Gazes Observed in Monocular Video

Cited by 19 publications

References 59 publications

Extended Gaze Following: Detecting Objects in Videos Beyond the Camera Field of View

Extended Gaze Following: Detecting Objects in Videos Beyond the Camera Field of View

When I Look into Your Eyes: A Survey on Computer Vision Contributions for Human Gaze Estimation and Tracking

LAEO-Net: Revisiting People Looking at Each Other in Videos

Contact Info

Product

Resources

About