A unified evaluation framework for head motion prediction methods in 360° videos

Rondón, Miguel Fabián Romero; Sassatelli, Lucile; Aparicio-Pardo, Ramón; Precioso, Fŕed́eric

doi:10.1145/3339825.3394934

Cited by 16 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, the horizontal bias in VR only varied with mask types by its amplitude. We have shown that the head generally spans scenes with movements long in duration and amplitudes; consequently, future head movements during free-viewing can be predicted with a time-dependent model ( Nguyen et al., 2018 ; Rondón et al., 2020 ). Because in the displays’ frame of reference the eyes do not move significantly far from the centre ( Sitzmann et al., 2017 ), a big part of the peripheral content presented in the headset could be decreased in quality as a function of eccentricity, foveated rendering would allow compressing visual information further ( Weier et al., 2017 ).…”

Section: Discussionmentioning

confidence: 99%

What are the visuo-motor tendencies of omnidirectional scene free-viewing in virtual reality?

et al. 2022

View full text Add to dashboard Cite

Central and peripheral vision during visual tasks have been extensively studied on two-dimensional screens, highlighting their perceptual and functional disparities. This study has two objectives: replicating on-screen gaze-contingent experiments removing central or peripheral field of view in virtual reality, and identifying visuo-motor biases specific to the exploration of 360 scenes with a wide field of view. Our results are useful for vision modelling, with applications in gaze position prediction (e.g., content compression and streaming). We ask how previous on-screen findings translate to conditions where observers can use their head to explore stimuli. We implemented a gaze-contingent paradigm to simulate loss of vision in virtual reality, participants could freely view omnidirectional natural scenes. This protocol allows the simulation of vision loss with an extended field of view ( 80°) and studying the head's contributions to visual attention. The time-course of visuo-motor variables in our pure free-viewing task reveals long fixations and short saccades during first seconds of exploration, contrary to literature in visual tasks guided by instructions. We show that the effect of vision loss is reflected primarily on eye movements, in a manner consistent with two-dimensional screens literature. We hypothesize that head movements mainly serve to explore the scenes during free-viewing, the presence of masks did not significantly impact head scanning behaviours. We present new fixational and saccadic visuo-motor tendencies in a 360° context that we hope will help in the creation of gaze prediction models dedicated to virtual reality.

show abstract

Section: Discussionmentioning

confidence: 99%

What are the visuo-motor tendencies of omnidirectional scene free-viewing in virtual reality?

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The settings of the federated learning are as follows. In each round, all of K = 30 users update 2 According to the analysis in [2], [20], the traces of the first 20 users in the dataset have mistakes, thus we only use the traces of the other 30 users.…”

Section: Trace-driven Simulation Resultsmentioning

confidence: 99%

“…Spherical distance, known as orthodromic distance [2], [20], has been considered as the most appropriate metric to measure the viewpoint prediction error of FoV on the unit sphere [2]. As shown in Fig.…”

Section: E Prediction Error: Spherical Distancementioning

confidence: 99%

Privacy Leakage in Proactive VR Streaming: Modeling and Tradeoff

Wang¹,

Yang²,

Sun³

2022

Preprint

View full text Add to dashboard Cite

Proactive tile-based virtual reality (VR) video streaming employs the trace of viewpoint of a user to predict future requested tiles, then renders and delivers the predicted tiles before playback. Recently, it has been found that the identity and preference of the user can be inferred from the viewpoint data uploaded in proactive streaming procedure. When considering viewpoint leakage, several fundamental questions arise. When is the viewpoint leaked? Can privacy-preserving approaches, e.g., federated or individual training, or using local predicting and predictors with no need for training avoid viewpoint leakage? In this paper, we investigate viewpoint leakage during proactive streaming. We find that if the prediction error or the quality of experience (QoE) metric is uploaded for adaptive streaming, the real viewpoint can be inferred even with the above privacy-preserving approaches. Then, we define viewpoint leakage probability to characterize the inference accuracy of the real viewpoint, and respectively derive the probability when uploading prediction error and the QoE. We find there is a conditional tradeoff between viewpoint leakage probability and prediction performance, QoE, or the resources. Simulation with the state-of-the-art predictor over a real dataset shows that the tradeoff between viewpoint privacy and prediction performance, resources, and QoE exist for most cases.

show abstract

“…Thus, it is necessary to predict and cache the tile files that users may watch in advance, which requires the prediction of the users' head motion trajectory in the coming period. Related research [27] shows that various head motion prediction models have been proposed to predict short-term head trajectory changes with high accuracy in recent years. In the tile-based prediction model presented in the literature [5], prediction accuracy can reach 84.22% when the prediction period is set to 1s.…”

Section: Introductionmentioning

confidence: 99%

Multi-MEC Cooperation Based VR Video Transmission and Cache using K-Shortest Paths Optimization

Xia¹,

Chen²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

In recent network architectures, multi-MEC cooperative caching has been introduced to reduce the transmission latency of VR videos, in which MEC servers' computing and caching capability are utilized to optimize the transmission process. However, many solutions that use the computing capability of MEC servers ignore the additional arithmetic power consumed by the codec process, thus making them infeasible. Besides, the minimum cache unit is usually the entire VR video, which makes caching inefficient. To address these challenges, we split VR videos into tile files for caching based on the current popular network architecture and provide a reliable transmission mechanism and an effective caching strategy. Since the number of different tile files N is too large, the current cooperative caching algorithms do not cope with such large-scale input data. We further analyze the problem and propose an optimized k-shortest paths (OKSP) algorithm with an upper bound time complexity of O((K •M +N )•M •log N )), and suitable for shortest paths with restricted number of edges, where K is the total number of tiles that all M MEC servers can cache in the collaboration domain. And we prove the OKSP algorithm can compute the caching scheme with the lowest average latency in any case, which means the solution given is the exact solution. The simulation results show that the OKSP algorithm has excellent speed for solving large-scale data and consistently outperforms other caching algorithms in the experiments.

show abstract

A unified evaluation framework for head motion prediction methods in 360° videos

Cited by 16 publications

References 11 publications

What are the visuo-motor tendencies of omnidirectional scene free-viewing in virtual reality?

What are the visuo-motor tendencies of omnidirectional scene free-viewing in virtual reality?

Privacy Leakage in Proactive VR Streaming: Modeling and Tradeoff

Multi-MEC Cooperation Based VR Video Transmission and Cache using K-Shortest Paths Optimization

Contact Info

Product

Resources

About