Interactive free viewpoint video streaming using prioritized network coding

Toni, Laura; Thomos, Nikolaos; Frossard, Pascal

doi:10.1109/mmsp.2013.6659330

Cited by 17 publications

(23 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Others have considered coding or network coding solutions for reduced bandwidth usage in multi-user scenarios [32], [33]. Both multicast-based delivery protocols [34] and peer-assisted solutions [35], [36] have also been proposed, typically delivering different views of a fixed encoding quality per stream over parallel overlay networks, but also solutions that implement pan/tilt/zoom functionality into the delivery protocol, taking into account the region of interest [37].…”

Section: H Bandwidth Variationsmentioning

confidence: 99%

Optimized Adaptive Streaming of Multi-video Stream Bundles

Carlsson

Eager

Krishnamoorthi

et al. 2017

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Abstract-In contrast to traditional video, multi-view video streaming allows viewers to interactively switch among multiple perspectives provided by different cameras. One approach to achieving such a service is to encode the video from all of the cameras into a single stream, but this has the disadvantage that only a portion of the received video data will be used, namely that required for the selected view at each point in time. In this paper we introduce the concept of a "multi-video stream bundle" that consists of multiple parallel video streams that are synchronized in time, each providing the video from a different camera capturing the same event or movie. For delivery we leverage the adaptive features and time-based chunking of HTTPbased Adaptive Streaming (HAS), but now employing adaptation in both content and rate. Users are able to change their viewpoint on-demand and the client player adapts the rate at which data is retrieved from each stream based on the user's current view, the probabilities of switching to other views, and the user's current bandwidth conditions. A crucial component of such a system is the prefetching policy. For this we present an optimization model as well as a simpler heuristic that can balance the playback quality and the probability of playback interruptions. After analytically and numerically characterizing the optimal solution, we present a prototype implementation and sample results. Our prefetching and buffer management solution is shown to provide close to seamless playback switching when there is sufficient bandwidth to prefetch the parallel streams.

show abstract

Section: H Bandwidth Variationsmentioning

confidence: 99%

Optimized Adaptive Streaming of Multi-video Stream Bundles

Carlsson

Eager

Krishnamoorthi

et al. 2017

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…The work in [29] has later been extended to the case of multi-view video in [41]. Cameras' streams are organized into layered subsets, with subsets organized based on their priority levels.…”

Section: Related Workmentioning

confidence: 99%

“…In our scheme, the different treatment of layers is not intended to differentiate the likelihood of their reception but rather the delay experienced by the user before they can start displaying it. For this reason, while the network coding scheme used in [41] varies the number of packets from each layer in the coding window, in our scheme, all packets from lower layers are introduced in the coding window before any packet of a higher layer is introduced.…”

Section: Related Workmentioning

confidence: 99%

Rate-distortion-optimized multi-view streaming in wireless environment using network coding

Greco¹,

Nemoianu

Cagnazzo

et al. 2016

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

Multi-view video streaming is an emerging video paradigm that enables new interactive services, such as 3D video, free viewpoint television, and immersive teleconferencing. Because of the high bandwidth cost they come with, multi-view streaming applications can greatly benefit from the use of network coding, in particular in transmission scenarios such as wireless network, where the channels have limited capacity and are affected by losses. In this paper, we address the topic of cooperative streaming of multi-view video content, wherein users who recently acquired the content can contribute parts of it to their neighbors by providing linear combinations of the video packets. We propose a novel method for selection and network encoding of the transmitted frames based on the users' preferences for the different views and the rate-distortion properties of the stream. Using network coding enables the users to retrieve the content in a faster and more reliable manner and without the need for coordination among the senders. Our experimental results prove that our preference-based approach provides a high-quality decoding even when the uplink capacity of each node is only a small fraction of the rate of the stream.

show abstract

“…Specifically, a virtual image of viewpoint u can be synthesized at decoder via depth-image-based rendering (DIBR) [1] using texture and depth maps of left and right reference views vL and vR, where vL ≤ u ≤ vR. Many practical challenges in interactive free viewpoint systems have already been addressed in works such as [2,3,4,5,6,7,8,9], which propose novel coding and/or streaming strategies at the server to optimize the tradeoff between transmission rate and visual distortion, while providing the promised viewnavigation ability at the client. As an example, in order to deal with network delay, the system in [3] transmits a subset of camera views enabling synthesis of a window W of possible virtual views (with the window size proportional to the round trip time (RTT) between server and client), so that the client can freely navigate in the 3D scene.…”

Section: Introductionmentioning

confidence: 99%

In-network view re-sampling for interactive free viewpoint video streaming

Toni

Cheung

Frossard

2015

2015 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

Interactive free viewpoint video offers the possibility for each user to independently choose the views of a 3D scene to be displayed at the decoder. The visual content is commonly represented by N texture and depth map pairs that capture different viewpoints. A server selects an appropriate subset of M ≤ N views for transmission, so that the user can freely navigate in the corresponding window of viewpoints without being affected by network delay. During navigation, a user can synthesize any intermediate virtual view image in the navigation window via depth-image-based rendering (DIBR) using two nearby camera views as references. When the available bandwidth is too small to transmit all camera views typically used to synthesize views in the navigation window, we propose to synthesize intermediate virtual views as new references for transmission-a resampling of viewpoints for the 3D scene-so that the synthesized view distortion within the navigation window is minimized. We formulate a combinatorial optimization problem to find the best set of M virtual views to synthesize as new references, and show that the problem is NP-hard. We approximate the original problem with a new reference view equivalence model and derive in this case an optimal dynamic programming algorithm to determine the best set of M views to be transmitted to each user. Experimental results show that synthesizing virtual views as new references for client-side view synthesis can outperform simple selection from camera views by up to 0.73dB in synthesized view quality.

show abstract

Interactive free viewpoint video streaming using prioritized network coding

Cited by 17 publications

References 16 publications

Optimized Adaptive Streaming of Multi-video Stream Bundles

Optimized Adaptive Streaming of Multi-video Stream Bundles

Rate-distortion-optimized multi-view streaming in wireless environment using network coding

In-network view re-sampling for interactive free viewpoint video streaming

Contact Info

Product

Resources

About