A high-quality low-delay remote rendering system for 3D video

Shi, Shu; Kamali, Mahsa; Nahrstedt, Klara; Hart, John C.; Campbell, Roy H.

doi:10.1145/1873951.1874011

Cited by 16 publications

(13 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To reduce these errors, multiple reference images [24] can be taken from different views and warped to the same target view. These reference images must be carefully chosen [3,14,24,37] so that mixing their warped views compensates the missing information in single frames. On top of using warping from multiple sprites, we filter sprites to be warped depending on views to ensure coherent lighting for the target view (see Sect.…”

Section: Image-based Renderingmentioning

confidence: 99%

“…Hudson and Mark [14] proposed to use three sets of reference images surrounding the user's view, warping twelve images per view, while some of the images may not be useful. Shi et al [37] proposed to predict a future reference view to be used with a main one so that the warping errors can be compensated. The quality of the prediction, however, depends on the actual future camera motion.…”

Section: Remote Renderingmentioning

confidence: 99%

“…Different from the above work, we propose to store many reference images in terms of distinctive sprites in a sprite tree and select carefully only those sprites useful to the target view for warping; the sprite filtering both improves the rendering quality and limits the complexity of the warping computation. The reference views can come from the user movement or from previously well-chosen reference views [3,14,24,37]. As image samples accumulated in a sprite tree, there are many more images to choose from for fewer occlusion errors, compared to a few chosen reference images.…”

Section: Remote Renderingmentioning

confidence: 99%

See 2 more Smart Citations

Sprite tree: an efficient image-based representation for networked virtual environments

et al. 2016

View full text Add to dashboard Cite

We propose a new and efficient image-based representation for networked virtual environments, called the sprite tree. A sprite tree organizes multiple reference images efficiently and compactly for accelerating the rendering of complex virtual scenes. Using our basic construction and rendering methods, the results show that a sprite tree can efficiently organize the pixels from hundreds of distinctive reference images and accelerate the rendering of a complex scene. Furthermore, we propose the sprite view similarity to (i) largely reduce the lighting artifacts in the rendered images, and (ii) significantly reduce the redundancy and the tree size with little loss of the visual quality.

show abstract

Section: Image-based Renderingmentioning

confidence: 99%

Section: Remote Renderingmentioning

confidence: 99%

Section: Remote Renderingmentioning

confidence: 99%

See 1 more Smart Citation

Sprite tree: an efficient image-based representation for networked virtual environments

et al. 2016

View full text Add to dashboard Cite

show abstract

“…In [43], the system contains intermediary servers that performs the virtual view rendering in the place of the light decoders and transmit the resulting images to decoders. However, this approach leads to high processing delays which can be addressed by choosing the appropriate remote rendering systems [44]. The SyncCast system [45] moreover enables the user to interact with each other for improved decoding performance.…”

Section: Related Workmentioning

confidence: 99%

Interactive Multiview Video System With Low Complexity 2D Look Around at Decoder

Maugey

Frossard

2013

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Abstract-Multiview video with interactive 2D look around at the receiver is a challenging application with several issues in terms of effective use of storage and bandwidth resources, reactivity of the system, quality of the viewing experience and system complexity. The impression of 3D immersion is highly dependent on the smoothness of the navigation and thus on the number of 2D viewpoints. The classical decoding system for generating virtual views first projects a reference or encoded frame to a given viewpoint and then fills in the holes due to potential occlusions. This last step still constitutes a complex operation with specific software or hardware at the receiver and requires a certain quantity of information from the neighboring frames for ensuring consistency between the virtual images. In this work we propose a new approach that shifts most of the burden due to interactivity from the decoder to the encoder, by anticipating the navigation of the decoder and sending auxiliary information that guarantees temporal and interview consistency. This leads to an additional cost in terms of transmission rate and storage, which we minimize by using optimization techniques based on the user behavior modeling. We show by experiments that the proposed system represents a valid solution for interactive multiview systems with classical decoders.Index Terms-2D look around viewing, interactivity, multiview video coding, view synthesis.

show abstract

“…The survey analyzes the interaction latency of each remote rendering system design and the details are summarized in [4]. Our work on 3D video rendering [4][5] answers the question (1) and (2) as shown above. For each 3D video frame, two or more depth images are generated on the rendering server as the reference frame.…”

Section: State Of Researchmentioning

confidence: 99%

Reduce latency: The key to successful interactive remote rendering systems

Shi

2011

2011 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops)

Self Cite

View full text Add to dashboard Cite

Remote rendering is considered as a general and effective approach to collaborate mobile devices and cloud computing services and deliver computation or network bandwidth intensive media content (e.g., 3D games, 3D video) to mobile devices. In this abstract, I propose my doctoral thesis research on interaction latency reduction for remote rendering systems. My work mainly focuses on how image based rendering techniques can be appropriately applied to reduce the interaction latency which is originally caused by network delay.

show abstract

A high-quality low-delay remote rendering system for 3D video

Cited by 16 publications

References 16 publications

Sprite tree: an efficient image-based representation for networked virtual environments

Sprite tree: an efficient image-based representation for networked virtual environments

Interactive Multiview Video System With Low Complexity 2D Look Around at Decoder

Reduce latency: The key to successful interactive remote rendering systems

Contact Info

Product

Resources

About