Correlation-Aware Packet Scheduling in Multi-Camera Networks

Abstract

In multiview applications, multiple cameras acquire the same scene from different viewpoints and generally produce correlated video streams. This results in large amounts of highly redundant data. In order to save resources, it is critical to handle properly this correlation during encoding and transmission of the multiview data. In this work, we propose a correlation-aware packet scheduling algorithm for multi-camera networks, where information from all cameras are transmitted over a bottleneck channel to clients that reconstruct the multiview images. The scheduling algorithm relies on a new rate-distortion model that captures the importance of each view in the scene reconstruction. We propose a problem formulation for the optimization of the packet scheduling policies, which adapt to variations in the scene content. Then, we design a low complexity scheduling algorithm based on a trellis search that selects the subset of candidate packets to be transmitted towards effective multiview reconstruction at clients. Extensive simulation results confirm the gain of our scheduling algorithm when inter-source correlation information is used in the scheduler, compared to scheduling policies with no information about the correlation or non-adaptive scheduling policies. We finally show that increasing the optimization horizon in the packet scheduling algorithm improves the transmission performance, especially in scenarios where the level of correlation rapidly varies with time.

Reference View Selection in DIBR-Based Multiview Coding

“…The similarity between views can be related to the image region that is common in two views. This concept introduced in [30] holds under two hypotheses that are commonly used in multiview video applications: i) the scene is lambertian (i.e., a point in the 3D scene is observed under similar illumination conditions from the different viewpoints) and ii) the navigation path remains approximately at a uniform distance from the scene 2 . Equipped with the above notion, we introduce here the dissimilarity metric γ to measure the difference between views.…”

“…The proposed model Eq. (2) relies on the observation that the coding rate of a predicted view is roughly linearly dependent on the dissimilarity with respect to the reference views [30]. Hence, the knowledge of the geometry of the scene (e.g., the dissimilarity between the views) permits to model the coding rates for a given target distortion.…”

“…The views have sub-sampled irregularly. In particular, we have selected 25 views, namely, 1,5,9,13,17,18,21,24,27,30,33,36,37,39,41,42,43,44,45,46,47,48,49,50,51, in the dataset of [36]. We have also tested our solution on a challenging test sequence, called new Tsukuba [39].…”

Optimized Packet Scheduling in Multiview Video Navigation Systems

“…In our work, we rather optimize the experienced quality of interactive users and the expected quality variations perceived over likely navigation paths. Other works [2], [3] have studied the problem of source correlation aware transmission policy optimization for multiview scheduling. However, the interactivity of users has been neglected and only predefined coding strategies have been considered so far.…”

“…These two challenges have not been addressed together, to the best of our knowledge. Spatial correlation has been taken into account in multisource resource allocation strategies for sensor networks [1] and for more general wireless networks [2], [3]. Interactivity of users is however mostly overlooked in resource allocation solutions in the literature.…”