A dataset of head and eye movements for 360° videos

David, Erwan; Gutiérrez, Jesús; Coutrot, Antoine; Silva, Matthieu Perreira Da; Callet, Patrick Le

doi:10.1145/3204949.3208139

Cited by 164 publications

(91 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The DNN is trained offline with a simulator of our exact 360°streaming player. It is fed with the 1121 head motion traces (with a sample each 0.2s) from the open dataset [6]. The instantaneous reward function r (t ), t describing the segment playback time, is set to r (t ) = Qual F oV (t ) − γ t −T (t ) , where T (t ) is the time of the last snap triggered before t. Hyper-parameters are: buffer size is 4s, γ = 0.3, number of training epoch is 1000, number of processes (agents) is 8, fraction of data for training, validation and test is 70%, 10% and 20%, respectively, the number of units in each layer is 128.…”

Section: Building Blocksmentioning

confidence: 99%

See 1 more Smart Citation

User-Adaptive Editing for 360 degree Video Streaming with Deep Reinforcement Learning

Sassatelli

Winckler

Fisichella

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

The development through streaming of 360°videos is persistently hindered by how much bandwidth they require. Adapting spatially the quality of the sphere to the user's Field of View (FoV) lowers the data rate but requires to keep the playback buffer small, to predict the user's motion or to make replacements to keep the buffered qualities up to date with the moving FoV, all three being uncertain and risky. We have previously shown that opportunistically regaining control on the FoV with active attention-driving techniques makes for additional levers to ease streaming and improve Quality of Experience (QoE). Deep neural networks have been recently shown to achieve best performance for video streaming adaptation and head motion prediction. This demo presents a step ahead in the important investigation of deep neural network approaches to obtain user-adaptive and network-adaptive 360°video streaming systems. In this demo, we show how snap-changes, an attentiondriving technique, can be automatically modulated by the user's motion to improve the streaming QoE. The control of snap-changes is made with a deep neural network trained on head motion traces with the Deep Reinforcement Learning strategy A3C. CCS CONCEPTS • Human-centered computing → Virtual reality; • Networks → Network simulations; • Computing methodologies → Neural networks.

show abstract

Section: Building Blocksmentioning

confidence: 99%

“…The video is streamed from a laptop over WiFi. The visitor will first choose which video they prefer to watch, available from the open dataset in [6]. By changing deliberately their head motion speed, they will observe they get repositioned in front of new FoVs with different frequencies.…”

Section: Building Blocksmentioning

confidence: 99%

User-Adaptive Editing for 360 degree Video Streaming with Deep Reinforcement Learning

Sassatelli

Winckler

Fisichella

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…47 Moreover, when shown in equirectangular projection, the 360°contents are non-Euclidean. 48 To address these problems, many studies on 360°images/videos in viewing databases, 49,50 image quality assessment, 51 and saliency detection, 52 have been conducted in recent years.…”

Section: Related Saliency Modelsmentioning

confidence: 99%

Saliency detection based on structural dissimilarity induced by image quality assessment model

Mou

2019

J. Electron. Imag.

View full text Add to dashboard Cite

The distinctiveness of image regions is widely used as the cue of saliency. Generally, the distinctiveness is computed according to the absolute difference of features. However, according to the image quality assessment (IQA) studies, the human visual system is highly sensitive to structural changes rather than absolute difference. Accordingly, we propose the computation of the structural dissimilarity between image patches as the distinctiveness measure for saliency detection. Similar to IQA models, the structural dissimilarity is computed based on the correlation of the structural features. The global structural dissimilarity of a patch to all the other patches represents saliency of the patch. We adopted two widely used structural features, namely the local contrast and gradient magnitude, into the structural dissimilarity computation in the proposed model. Without any post-processing, the proposed model based on the correlation of either of the two structural features outperforms 11 state-of-the-art saliency models on three saliency databases. Source code of the proposed model can be downloaded at https://github.com/yangli-xjtu/SDS.

show abstract

“…Privacy & Security in Eye Tracking There is a growing concern in keeping eye movement data private and secure in both real-time applications [49], and published datasets [50]. Publicly available datasets release de-identified gaze data from individuals viewing VR videos [19], the social interactions of children with ASD [22], and individual responses to emotional content such as nude imagery and faces [69]. Sensitive information, such as personality traits [36] and neurological diagnoses [47], could be linked to individuals that contributed to the aggregate data.…”

Section: Introductionmentioning

confidence: 99%

The Security-Utility Trade-off for Iris Authentication and Eye Animation for Social Virtual Avatars

John

Jörg

Koppal

et al. 2020

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

The gaze behavior of virtual avatars is critical to social presence and perceived eye contact during social interactions in Virtual Reality. Virtual Reality headsets are being designed with integrated eye tracking to enable compelling virtual social interactions. This paper shows that the near infra-red cameras used in eye tracking capture eye images that contain iris patterns of the user. Because iris patterns are a gold standard biometric, the current technology places the user's biometric identity at risk. Our first contribution is an optical defocus based hardware solution to remove the iris biometric from the stream of eye tracking images. We characterize the performance of this solution with different internal parameters. Our second contribution is a psychophysical experiment with a same-different task that investigates the sensitivity of users to a virtual avatar's eye movements when this solution is applied. By deriving detection threshold values, our findings provide a range of defocus parameters where the change in eye movements would go unnoticed in a conversational setting. Our third contribution is a perceptual study to determine the impact of defocus parameters on the perceived eye contact, attentiveness, naturalness, and truthfulness of the avatar. Thus, if a user wishes to protect their iris biometric, our approach provides a solution that balances biometric protection while preventing their conversation partner from perceiving a difference in the user's virtual avatar. This work is the first to develop secure eye tracking configurations for VR/AR/XR applications and motivates future work in the area.

show abstract

A dataset of head and eye movements for 360° videos

Cited by 164 publications

References 18 publications

User-Adaptive Editing for 360 degree Video Streaming with Deep Reinforcement Learning

User-Adaptive Editing for 360 degree Video Streaming with Deep Reinforcement Learning

Saliency detection based on structural dissimilarity induced by image quality assessment model

The Security-Utility Trade-off for Iris Authentication and Eye Animation for Social Virtual Avatars

Contact Info

Product

Resources

About