Using distributed source coding and depth image based rendering to improve interactive multiview video access

Abstract

Multiple-views video is commonly believed to be the next significant achievement in video communications, since it enables new exciting interactive services such as free viewpoint television and immersive teleconferencing. However the interactivity requirement (i.e. allowing the user to change the viewpoint during video streaming) involves a trade-off between storage and bandwidth costs. Several solutions have been proposed in the literature, using redundant predictive frames, Wyner-Ziv frames, or a combination of them. In this paper, we adopt distributed video coding for interactive multiview video plus depth (MVD), taking advantage of depth image based rendering (DIBR) and depth-aided inpainting to fill the occlusion areas. To the authors' best knowledge, very few works in interactive MVD consider the problem of continuity of the playback during the switching among streams. Therefore we survey the existing solutions, we propose a set of techniques for MVD coding and we compare them. As main results, we observe that DIBR can help in rate reduction (up to 13.36% for the texture video and up to 8.67% for the depth map, wrt the case where DIBR is not used), and we also note that the optimal strategy to combine DIBR and distributed video coding depends on the position of the switching time into the group of pictures. Choosing the best technique on a frame-to-frame basis can further reduce the rate from 1% to 6%.

High Performance and Hardware Efficient Multiview Video Coding Frame Scheduling Algorithms and Architectures

“…2, we need to note that the frame B (1,0) is desired to be sent to the output frame at T1; however, this frame is encoded at T7 in RPFO. Therefore, the output of this frame has to be delayed until T7, while the output of B (1,0) in LPFO is delayed to T4. This implies that the FWT is dependent on the frame ordering scenario.…”

“…MVC uses more than two cameras during video acquisition, while stereoview video coding offers 3-D scenes using only two cameras. Due to more texture and depth information, MVC can reduce the occlusion problem [1], and can provide richer viewing experiences compared to traditional single-and stereo-view video coding [2].…”

A distributed video coding system for multi view video plus depth

“…Some works [9][10][11] about DVC applied to multiview video plus depth have only highlighted the advantage of sharing the motion information between texture and depth. In a previous work [12], DVC is applied to MVD in order to assure continuity of the playback in the context of the interactive multiview video streaming paradigm. Depth map can be used for generating the target view when a view switching occurs, in order to have an estimation of the WZF.…”

Interactive Multiview Video System With Low Complexity 2D Look Around at Decoder

“…The possible type of frames are intra frames and predicted frames (with the storage of different motion vectors and residuals). Petrazzuoli et al [40] have recently introduced the idea of using distributed source coding and inter-view prediction for effective multiview switching.…”