Adaptive Block-Based Compressed Video Sensing Based on Saliency Detection and Side Information

Abstract

The setting of the measurement number for each block is very important for a block-based compressed sensing system. However, in practical applications, we only have the initial measurement results of the original signal on the sampling side instead of the original signal itself, therefore, we cannot directly allocate the appropriate measurement number for each block without the sparsity of the original signal. To solve this problem, we propose an adaptive block-based compressed video sensing scheme based on saliency detection and side information. According to the Johnson–Lindenstrauss lemma, we can use the initial measurement results to perform saliency detection and then obtain the saliency value for each block. Meanwhile, a side information frame which is an estimate of the current frame is generated on the reconstruction side by the proposed probability fusion model, and the significant coefficient proportion of each block is estimated through the side information frame. Both the saliency value and significant coefficient proportion can reflect the sparsity of the block. Finally, these two estimates of block sparsity are fused, so that we can simultaneously use intra-frame and inter-frame correlation for block sparsity estimation. Then the measurement number of each block can be allocated according to the fusion sparsity. Besides, we propose a global recovery model based on weighting, which can reduce the block effect of reconstructed frames. The experimental results show that, compared with existing schemes, the proposed scheme can achieve a significant improvement in peak signal-to-noise ratio (PSNR) at the same sampling rate.

Lightweight Cross-Modal Information Mutual Reinforcement Network for RGB-T Salient Object Detection

“…It is important to note that SOD differs from object detection tasks that aim to predict object bounding boxes. SOD has been employed as a preprocessing step in many computer vision tasks, such as image fusion [1], perceptual video coding [2], compressed video sensing [3], image quality assessment [4], and so on.…”

NRVC: Neural Representation for Video Compression with Implicit Multiscale Fusion Network

“…In recent years, due to the powerful feature learning ability of deep networks and the robustness defects of manually designed traditional video compression technology modules [ 1 , 2 , 3 ], more and more researchers are no longer satisfied with changing a single module and have begun to focus on building an end-to-end video compression deep model [ 4 ]. By implementing all modules with deep neural networks and directly using end-to-end optimization rate-distortion objective functions, the global optimal solution can be obtained more quickly.…”