Advanced free viewpoint video streaming techniques

Abstract

Free-viewpoint video is a new type of interactive multimedia service allowing users to control their viewpoint and generate new views of a dynamic scene from any perspective. The uniquely generated and displayed views are composed from two or more high bitrate camera streams that must be delivered to the users depending on their continuously changing perspective. Due to significant network and computational resource requirements, we proposed scalable viewpoint generation and delivery schemes based on multicast forwarding and distributed approach. Our aim was to find the optimal deployment locations of the distributed viewpoint synthesis processes in the network topology by allowing network nodes to act as proxy servers with caching and viewpoint synthesis functionalities. Moreover, a predictive multicast group management scheme was introduced in order to provide all camera views that may be requested in the near future and prevent the viewpoint synthesizer algorithm from remaining without camera streams. The obtained results showed that even 42% traffic decrease can be realized using distributed viewpoint synthesis and the probability of viewpoint synthesis starvation can be also significantly reduced in future free viewpoint video services.

Edge-FVV: Free Viewpoint Video Streaming by Learning at the Edge

“…But in practice, as the number of users increases, the growing number of virtual view synthesis tasks will place a huge burden on the server or cost extra computational resources on the users' devices. Hence, a distributed system is needed where the synthesis task is conducted at the network edge (e.g., proxy, edge cache, edge server) [1], [2]. A distributed FVV system can focus on improving the navigation quality for the client, and on studying the reference view selection problem under limited bandwidth [2].…”

“…A distributed FVV system can focus on improving the navigation quality for the client, and on studying the reference view selection problem under limited bandwidth [2]. The network architecture of an FVV system can be abstracted as a k-ary tree with the main server at the root and the users at the leaves, where the system calculates the total bandwidth cost to optimize the selected location of the edge servers [1]. In this work, we study the impact of the number of edge caches on the delay for a user's request, and seek to optimize the delay by allocating users to caches.…”

“…While FVV acquisition can be completed offline, the virtual view synthesis protocol may consume significant bandwidth and computing resources. To alleviate these burdens, existing research [1], [2] proposes adding extra edge caches (located between the server and the users) to undertake the virtual view synthesis task, given a requested viewpoint. The videos of reference viewpoints can be stored in the edge caches to accelerate the process for incoming requests to the given viewpoint.…”

Random access prediction structures for light field video coding with MV-HEVC

“…For a video to be fully random accessed in time for example, all video frames must be able to be independently decoded, and consequently independently accessed. Besides random access in time, VR and free viewpoint television (FTV) applications also have view random access as a crucial requirement for delivering high quality content at any possible instance both in time and in space (free navigation) [8,9]. For this work, we define view random access as the ability of a decoder to switch to a different view immediately at any point in time.…”