SVP: Sinusoidal Viewport Prediction for 360-Degree Video Streaming

Abstract

The rapid growth of user expectations and network technologies has proliferated the service needs of 360-degree video streaming. In the light of the unprecedented bitrates required to deliver entire 360-degree videos, tile-based streaming, which associates viewport and non-viewport tiles with different qualities, has emerged as a promising way to facilitate 360-degree video streaming in practice. Existing work on viewport prediction primarily targets prediction accuracy, which potentially gives rise to excessive computational overhead and latency. In this paper, we propose a sinusoidal viewport prediction (SVP) system for 360-degree video streaming to overcome the aforementioned issues. In particular, the SVP system leverages 1) sinusoidal values of rotation angles to predict orientation, 2) the relationship between prediction errors, prediction time window and head movement velocities to improve the prediction accuracy, and 3) the normalized viewing probabilities of tiles to further improve adaptive bitrate (ABR) streaming performance. To evaluate the performance of the SVP system, we conduct extensive simulations based on real-world datasets. Simulation results demonstrate that the SVP system outperforms state-of-the-art schemes under various buffer thresholds and bandwidth settings in terms of viewport prediction accuracy and video quality, revealing its applicability to both live and video-on-demand streaming in practical scenarios. INDEX TERMS 360-degree video, viewport prediction, live streaming, video on demand.

Lightweight Neural Network-Based Viewport Prediction for Live VR Streaming in Wireless Video Sensor Network

“…Nasrabadi et al [5] proposed a clustering-based method to estimate the user's future viewport. Jiang et al [6] interpreted the original rotation amount as a sine value to reduce the prediction error of the yaw direction. ese trajectory-based methods may be used in live because they only require real-time data from the current video session.…”

“…e most advanced VR streaming research mainly focuses on viewport prediction [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16]. e existing solutions [1,[11][12][13] suggest prefetching all the tiles of each segment, and higher quality of prefetching predicts the tiles in the viewport.…”

“…e existing solutions [1,[11][12][13] suggest prefetching all the tiles of each segment, and higher quality of prefetching predicts the tiles in the viewport. Most existing viewport prediction algorithms can be divided into trajectory-based [1][2][3][4][5][6] and content-based [7][8][9][10][11][12][13][14][15][16] methods. However, these methods are either difficult to achieve sufficient prediction accuracy, or they occupy a large amount of server resources in actual deployment and cannot achieve excellent real-time performance due to the complexity of their own algorithms.…”

360Cast+: Viewport Adaptive Soft Delivery for 360-Degree Videos

“…The quality of the viewport-based streaming highly depends on the accuracy of the viewport prediction. For accurate viewport prediction, there are non-learning-based [35]- [39] and learning-based [40]- [43] prediction methods. The most existing studies on the non-learning-based viewport prediction utilize a regression curve obtained from the past head movement, such as linear regression (LR) [35], [38], [39] and weighted LR (WLR) [36], [37].…”

“…In the learning-based prediction methods, [40] used saliency map prediction based on orientation data of multiple users and [41] further adopted the recurrent neural network (RNN)-based long short-term memory (LSTM) model to predict future viewport movement from both the saliency map and past head orientation. Some studies [42], [43] discuss the effect of the LR-based and deeplearning-based methods on the viewport prediction. They found that the difference of the prediction accuracy between both methods is slight.…”