Bitrate Efficient 3DoF+ 360 Video View Synthesis for Immersive VR Video Streaming

Jeong, Jong-Beom; Jang, Dongmin; Son, Jangwoo; Ryu, Eun‐Seok

doi:10.1109/ictc.2018.8539414

Cited by 13 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The previous our mobile VR streaming projects contribute 360 video streaming can be approached in a limited performance of mobile VR in [13][14][15][16][17]. Additionally, we conducted studies regarding native VR [1][2][3]. These experiences enabled us to conduct an improved study on 360 video tile-based streaming for Figure 1: Conceptual architecture of proposed system a native VR system connected to a PC based on the advantages of HEVC coding.…”

Section: Tile-based Streaming Of 360 Videosmentioning

confidence: 99%

“…Certain standards, such as 3DoF, 3DoF+, and 6DoF, have been developed, typically for enhancing user experiences with regard to activities such as moving one's head, turning, and walking in a VR environment. Few studies [1,2] improved 3DoF+ and 6DoF to reduce video quality by employing various methods such as size reduction and the elimination of correlations between videos [3].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Efficient Viewport-Dependent 360 VR System Based on Adaptive Tiled Streaming

Le¹,

Jeong²,

Lee³

et al. 2021

Computers, Materials &Amp; Continua

Self Cite

View full text Add to dashboard Cite

Recent advances in 360 video streaming technologies have enhanced the immersive experience of video streaming services. Particularly, there is immense potential for the application of 360 video encoding formats to achieve highly immersive virtual reality (VR) systems. However, 360 video streaming requires considerable bandwidth, and its performance depends on several factors. Consequently, the optimization of 360 video bitstreams according to viewport texture is crucial. Therefore, we propose an adaptive solution for VR systems using viewport-dependent tiled 360 video streaming. To increase the degrees of freedom of users, the moving picture experts group (MPEG) recently defined three degrees plus of freedom (3DoF+) and six degrees of freedom (6DoF) to support free user movement within camera-captured scenes. The proposed method supports 6DoF to allow users to move their heads freely. Herein, we propose viewport-dependent tiled 360 video streaming based on users' head movements. The proposed system generates an adaptive bitstream using tile sets that are selected according to a parameter set of user's viewport area. This extracted bitstream is then transmitted to the user's computer. After decoding, the user's viewport is generated and rendered on VR head-mounted display (HMD). Furthermore, we introduce certain approaches to reduce the motion-to-photon latency. The experimental results demonstrated that, in contrast with non-tiled streaming, the proposed method achieved high-performance 360 video streaming for VR systems, with a 25.89% BD-rate saving for Y-PSNR and 61.16% for decoding time.

show abstract

Section: Tile-based Streaming Of 360 Videosmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Efficient Viewport-Dependent 360 VR System Based on Adaptive Tiled Streaming

Le¹,

Jeong²,

Lee³

et al. 2021

Computers, Materials &Amp; Continua

Self Cite

View full text Add to dashboard Cite

show abstract

“…A promising approach that meets these specific requirements is necessary. Various depth map coding related research works have been proposed for depth estimation [92], view synthesis [93], bitrate reduction [94,95], and others [90,96,97] in 3DoF+/6DoF realisation, but they are still at the very early stage of technological development. MPEG is still developing the standard for immersive media.…”

Section: Role Of Depth Coding In Future Immersive Mediamentioning

confidence: 99%

Overview of current development in depth map coding of 3D video and its future

Chan

Chen

et al. 2020

IET signal process.

View full text Add to dashboard Cite

3D videos have attracted attention from academia and industry after great success in the film industry. Multiview video plus depth (MVD) is the most popular 3D video format to provide vivid 3D feeling and has been adopted as an international 3D video coding standard, namely 3D extension of high efficiency video coding (HEVC). MVD includes a limited number of textures and depth maps to synthesise virtual views. In MVD, depth samples describe the distance between a camera and an actual object as a grey-level image. The characteristics of depth maps are quite different from texture images. Consequently, new coding tools are designed for depth maps in 3D-HEVC to improve the coding efficiency at the expense of high-computational complexity, which faces great challenges in coding systems. Depth map coding is also an important technique in immersive media to support three degrees of freedom 3DoF+/6DoF applications such as virtual reality/augmented reality. The study starts with an overview of what has been done over the last decade in 3D-HEVC, especially depth map coding, regarding theories, methodologies, current research and state-of-the-art fast approaches. Following this, a comprehensive comparison of the reviewed techniques is presented, and an outlook on its future trends is provided.

show abstract

“…Although this is the most common type of omnidirectional video content, it is not enough to provide a full immersive experience. In fact, a so-called 3DoF+ concept has been described as an evolution from the 3DoF feature [11]. It allows limited movements along the yaw, pitch and roll axis, which requires the content to have additional depth and texture information.…”

Section: A Omnidirectional Video Contentmentioning

confidence: 99%

Enhancing the Broadcasted TV Consumption Experience With Broadband Omnidirectional Video Content

et al. 2019

View full text Add to dashboard Cite

The current wide range of heterogeneous consumption devices and delivery technologies, offers the opportunity to provide related contents in order to enhance and enrich the TV consumption experience. This paper describes a solution to handle the delivery and synchronous consumption of traditional broadcast TV content and related broadband omnidirectional video content. The solution is intended to support both hybrid (broadcast/broadband) delivery technologies and has been designed to be compatible with the Hybrid Broadcast Broadband TV (HbbTV) standard. In particular, some specifications of HbbTV, such as the use of global timestamps or discovery mechanisms, have been adopted. However, additional functionalities have been designed to achieve accurate synchronization and to support the playout of omnidirectional video content in current consumption devices. In order to prove that commercial hybrid environments could be immediately enhanced with this type of content, the proposed solution has been included in a testbed, and objectively and subjectively evaluated. Regarding the omnidirectional video content, the two most common types of projections are supported: equirectangular and cube map. The results of the objective assessment show that the playout of broadband delivered omnidirectional video content in companion devices can be accurately synchronized with the playout on TV of traditional broadcast 2D content. The results of the subjective assessment show the high interest of users in this type of new enriched and immersive experience that contributes to enhance their Quality of Experience (QoE) and engagement.

show abstract

Bitrate Efficient 3DoF+ 360 Video View Synthesis for Immersive VR Video Streaming

Cited by 13 publications

References 6 publications

An Efficient Viewport-Dependent 360 VR System Based on Adaptive Tiled Streaming

An Efficient Viewport-Dependent 360 VR System Based on Adaptive Tiled Streaming

Overview of current development in depth map coding of 3D video and its future

Enhancing the Broadcasted TV Consumption Experience With Broadband Omnidirectional Video Content

Contact Info

Product

Resources

About