Wyner-ziv coding of light fields for random access

Aaron, Anne; Ramanathan, P.; Girod, Bernd

doi:10.1109/mmsp.2004.1436558

Cited by 41 publications

(50 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interactive media streaming has the advantage of reduced bandwidth utilization since only the requested media subset is transmitted. It is used for a wide range of media modalities, such as interactive light field [13], interactive image browsing [14], flexible video playback [15]. For multiview video, MVC [4], [16] discussed in the Introduction where multiple captured views are compressed efficiently together into a single stream would be suitable for non-interactive media streaming.…”

Section: A Interactive and Non-interactive Media Streamingmentioning

confidence: 99%

Delay-Cognizant Interactive Streaming of Multiview Video With Free Viewpoint Synthesis

Xiu

Cheung

Liang

2012

IEEE Trans. Multimedia

View full text Add to dashboard Cite

In interactive multiview video streaming (IMVS), a client receives and observes one of many available viewpoints of the same scene, and periodically requests from server view-switches to neighboring views, as the video is played back in time uninterruptedly. One key technical challenge is to design a frame coding structure that facilitates periodic view-switching, and achieves an optimal tradeoff between storage cost and expected transmission rate. In this paper, we first propose three significant improvements over existing IMVS system, and then study the corresponding frame structure optimization. First, using depth-image-based rendering, the new IMVS system enables free viewpoint switching, i.e., by encoding and transmitting both texture and depth maps of captured views, a client can select and synthesize any virtual view from an almost continuum of viewpoints between the left-most and right-most captured views. Second, IMVS system adopts a more realistic Markovian view-switching model with memory that more accurately captures user behaviors than previous memoryless models. View-switching model is used in predicting client's future view-switching patterns. Third, assuming that the round-trip-time (RTT) delay during server-client communication is non-negligible, during an IMVS session, IMVS system additionally transmits redundant frames RTT into future playback, so that zero-delay view-switching can be achieved. Given these improvements, we formalize a new join optimization of the frame coding structure, transmission schedule, and quantization parameters of the texture and depth maps of multiple camera views. We propose an iterative algorithm to achieve fast and near-optimal solutions.The convergence of the algorithm is also demonstrated. Experimental results show that the proposed optimized rate allocation method requires 38% less transmission rate than the fixed rate allocation scheme. In addition, with the same storage, the transmission rate of the optimized frame structure can be up to 55% lower than that of I-frame-only structure, and 27% lower than that of the structure without distributed source coding (DSC) frames. Index Termsmultiview video, video streaming, media interaction, view synthesis

show abstract

Section: A Interactive and Non-interactive Media Streamingmentioning

confidence: 99%

Delay-Cognizant Interactive Streaming of Multiview Video With Free Viewpoint Synthesis

Xiu

Cheung

Liang

2012

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…Early coding structure proposals to address this difficulty include [6,7]. [6] assumed a user only switches to an adjacent view during an ILFS session, and hence one out of a small subset of adjacent frames must be available at decoder for prediction of the target image during a view-switch.…”

Section: Related Workmentioning

confidence: 99%

“…For the same assumption of adjacent view-switches, [7] proposed to use DSC frames instead, where the number of leastsignificant-bit (LSB) bit-planes of transform coefficients that need to be transmitted depends on the quality of the side information, i.e., the correlation between the predictor frame at decoder and the target image. The key difference between [6,7] and our work is that we assume a much more general view-switching model for ILFS, where non-adjacent views can be selected by clients (see example user interface in [2] where non-adjacent views can be selected effortlessly).…”

Section: Related Workmentioning

confidence: 99%

“…The key difference between [6,7] and our work is that we assume a much more general view-switching model for ILFS, where non-adjacent views can be selected by clients (see example user interface in [2] where non-adjacent views can be selected effortlessly). Keeping server transmission rate low even for more generally accessed light field images is a new technical challenge.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Optimal frame structure design using landmarks for interactive light field streaming

Cai

Cheung

Lee

et al. 2012

2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Light field is a large set of spatially correlated images of the same static scene captured using a 2D array of closely spaced cameras. Interactive light field streaming is the application where a client continuously requests successive light field images along a view trajectory of his choosing, and in response the server transmits appropriate data for the client to correctly reconstruct desired images. The technical challenge is how to encode captured light field images into a reasonably sized frame structure a priori (without knowing eventual clients' view trajectories), so that at stream time, expected server transmission rate can be minimized, while satisfying client's viewswitch requests. In this paper, using I-frames, redundant P-frames and distributed source coding (DSC) frames as building blocks, we design coding structures to optimally trade off storage size of the frame structure with expected server transmission rate. The key novelty is to facilitate the use of "landmarks" in the structure-popular reference frames cached in the decoder buffer-so that the probability of having at least one useful predictor frame available in the buffer for disparity compensation is greatly increased. We first derive recursive equations to find the optimal caching strategy for a given coding structure. We then formulate the structure design problem as a Lagrangian minimization, and propose fast heuristics to find near-optimal solutions. Experimental results show that the expected server streaming rate can be reduced by up to 93.6% compared to an I-frame-only structure, at twice the storage required.Index Terms-light field, interactive streaming, optimization INTRODUCTION Light field[1] is a large set of spatially correlated images of the same static scene taken from a 2D array of closely spaced cameras. Because conventional display terminals show only one image at a time, a client typically browses the light field data by selecting single images in succession across time [2]. In a network streaming scenario then, a server will transmit pre-encoded images corresponding to client's successive view-switch requests along his chosen view trajectory. This network streaming service is called interactive light field streaming (ILFS) [3] in the literature.The technical challenge for ILFS is to encode captured light field images into a reasonably sized frame structure a priori, so that during actual streaming session, the expected server transmission rate to the client interactively selecting views is minimized. The problem is challenging because at encoding time, the exact view trajectory that a client will take at stream time is unknown, making it difficult to employ conventional differential coding to reduce the transmission rate. Differential coding, typical in compression of temporal frames in single-view video like H.263, assumes a previous decoded frameFt−1 of time instant t−1 is available at decoder for prediction of target image Ft of instant t, so that only (quantized) differential Ft −Ft−1 needs to be coded and...

show abstract

“…SP frames have been used for interactive streaming of static light fields [57,56]. Another solution to the random access problem associated with multi-view data-sets is based on distributed source coding (DSC) [32,7]. In this solution, an interframe coded picture is represented using enough parity bits which leads to an identical reconstruction irrespective of the reference frame used by the decoder.…”

Section: Multi-view Images/videosmentioning

confidence: 99%

Video Streaming with Interactive Pan/Tilt/Zoom

Mavlankar

Girod

2010

Signals and Communication Technology

View full text Add to dashboard Cite

High-spatial-resolution videos offer the possibility of viewing an arbitrary region-of-interest (RoI) interactively. The user can pan/tilt/zoom while watching the video. This chapter presents spatial-random-access-enabled video compression that encodes the content such that arbitrary RoIs corresponding to different zoom factors can be extracted from the compressed bit-stream. The chapter also covers RoI trajectory prediction, which allows pre-fetching relevant content in a streaming scenario. The more accurate the prediction the lower is the percentage of missing pixels. RoI prediction techniques can perform better by adapting according to the video content in addition to simply extrapolating previous moves of the input device. Finally, the chapter presents a streaming system that employs application-layer peer-to-peer (P2P) multicast while still allowing the users to freely choose individual RoIs. The P2P overlay adapts on-the-fly for exploiting the commonalities in the peers' RoIs. This enables peers to relay data to each other in real-time, thus drastically reducing the bandwidth required from dedicated servers.

show abstract

Wyner-ziv coding of light fields for random access

Cited by 41 publications

References 13 publications

Delay-Cognizant Interactive Streaming of Multiview Video With Free Viewpoint Synthesis

Delay-Cognizant Interactive Streaming of Multiview Video With Free Viewpoint Synthesis

Optimal frame structure design using landmarks for interactive light field streaming

Video Streaming with Interactive Pan/Tilt/Zoom

Contact Info

Product

Resources

About