Dynamic Viewport Selection-Based Prioritized Bitrate Adaptation for Tile-Based 360° Video Streaming

Abstract

Interactive 360°remote video applications have seen booming advancements due to the proliferation of smart display devices that enable a truly immersive experience. Compared to regular monoscopic videos, 360°videos have different requirements related to content preparation, packaging, transmission, specialized viewing equipment, and display factors (e.g., brightness, contrast, delay, frame rate, resolution, image quality, etc. In addition, 360°video requires substantial network and computational resources, which are challenging to achieve with conventional transmission and rendering infrastructure. Viewport-adaptive streaming is a common way to ensure visual quality under limited bandwidth resources. However, identifying, extracting, and rendering the true viewport in response to drastic head rotations can adversely affect user experience. This paper proposes two dynamic viewport selection approaches, which adapt the streamed regions based on content complexity variations and positional information to ensure viewport availability and smooth visual angles for VR users. They incorporate content information as well as user head movement patterns to support tile-based prioritized 360°video streaming. Moreover, a practical, prioritized bitrate adaptation approach, which requests selected tiles at appropriate quality levels, is also proposed to reduce the impact of inefficient bandwidth utilization in the VR scene. Experimental evaluations under real 4G bandwidth logs demonstrate that the proposed solutions outperform the closest state-of-theart algorithms across multiple performance metrics, i.e., viewport overlap, perceived quality levels, quality fluctuations, and viewport bandwidth utilization.

A Survey on Video Streaming for Next-Generation Vehicular Networks

“…Transmitting the entire video in high quality is evidently not wise, as users are typically limited to viewing only a small portion of the 360 • video. Viewport-adaptive streaming is a direct method for decreasing transmission bandwidth, dynamically adjusting the scene quality in real time based on the user's FoV [175]. After projection, 360 • videos are spatially divided into equally or unequally sized rectangles called tiles.…”

“…Yaqoob et al [175] considered two viewport prediction mechanisms when selecting tiles, adjusting the streaming region based on changes in content complexity and positional information. This adjustment ensures the availability of the viewport, providing a seamless perspective for 360 • video users.…”

TVSR‐OR: Tile‐based 360‐degree video streaming over real time streaming protocol with optimized read

“…Some tile‐based solutions have been proposed to reduce network bandwidth consumption. Examples can be seen in References 6,7,18‐25, to name but just a few. Zare et al proposed a tile‐based VR streaming system.…”

OJUMP: Optimization for joint unicast‐multicast panoramic VR live streaming system

“…Moreover, similar schemes using ABR algorithms to improve the delivery of VR video have also been proposed. [15][16][17][18] Besides, for 360 • video streams, considering the cooperative edge transcoding and caching problem in edge clusters, Yang et al 19 proposed a deep reinforcement learning-based stream caching and computing resource allocation scheme. Storage and computation resources are allocated jointly to improve cache hit rate and reduce latency and transmission cost.…”

“…At the same time, the algorithm is able to provide users with a high QoE. Moreover, similar schemes using ABR algorithms to improve the delivery of VR video have also been proposed 15–18 . Besides, for 360° video streams, considering the cooperative edge transcoding and caching problem in edge clusters, Yang et al 19 proposed a deep reinforcement learning‐based stream caching and computing resource allocation scheme.…”