Joint Compressive Video Coding and Analysis

Abstract

Traditionally, video acquisition, coding and analysis have been designed and optimized as independent tasks. This has a negative impact in terms of consumed resources, as most of the raw information captured by conventional acquisition devices is discarded in the coding phase, while the analysis step only requires a few descriptors of salient video characteristics. Recent Compressive Sensing literature has partially broken this paradigm by proposing to integrate sensing and coding in a unified architecture composed by a light encoder and a more complex decoder, which exploits sparsity of the underlying signal for efficient recovery. However, a clear understanding of how to embed video analysis in this scheme is still missing. In this paper, we propose a joint compressive video coding and analysis scheme and, as a specific application example, we consider the problem of object tracking in video sequences. We show that, weaving together compressive sensing and the information computed by the analysis module, the bit-rate required to perform reconstruction and tracking of the foreground objects can be considerably reduced, with respect to a conventional disjoint approach that postpones the analysis after the video signal is recovered in the pixel domain.These findings suggest that considerable gains in performance can be potentially obtained in video analysis applications, provided that a joint analysis-aware design of acquisition, coding and signal recovery is carried out.

Compressive Sensing in Electromagnetics - A Review

“…The promise to overcome the common wisdom in data acquisition based on Shannon's celebrated theorem [1] and to allow one to recover (certain) signals/phenomena from far fewer measurements than traditional techniques has attracted considerable interest (see Figure 1). Therefore, its study/application to those systems usually severely constrained by the Nyquist's sampling rate [1,2] (e.g., imaging [3,4], audio/video capture [5], and communications [6]) has been strongly motivated.…”

Compressive Video Sampling With Approximate Message Passing Decoding

“…The crucial idea of this method is that the motion estimation and CS sparsity-promoting reconstruction is carried out alternately in the multiscale framework. Cossalter et al [9] also considered the motion estimation in CS video analysis. Stankovic [30] and Prades-Nebot et al [25] divided each frame into non-overlapping blocks and approximate each block by a linear combination of blocks in previously transmitted frames the CS decoding process.…”

Compressive Sensing in Visual Tracking

“…Under such assumptions, this method is able to perform signal tracking with a low data rate and low computational complexity. (Cossalter et al, 2010) consider a collection of methods via which systems utilizing compressive imaging devices can perform visual tracking. Of particular note is a method referred to as joint compressive video coding and analysis, via which the tracker output is used to improve the overall effectiveness of the system.…”

“…If the weights are close to their ideal value (4.9), the value off t+1 obtained from the weighted decoding procedure will be of higher quality than that obtained from a more generic CS decoder. (Cossalter et al, 2010) explore two methods via which the weights at time t + 1 can be selected usingf t andẑ t . The best of these consists of three steps: 1) thresholding the entries off t , 2) translating the thresholded silhouettes for a single time step according to the motion model andẑ t , and 3) dilating the translated silhouettes using a predefined dilation element.…”