Content-Aware Scalable Deep Compressed Sensing

Abstract

To more efficiently address image compressed sensing (CS) problems, we present a novel content-aware scalable network dubbed CASNet which collectively achieves adaptive sampling rate allocation, fine granular scalability and high-quality reconstruction. We first adopt a data-driven saliency detector to evaluate the importances of different image regions and propose a saliency-based block ratio aggregation (BRA) strategy for sampling rate allocation. A unified learnable generating matrix is then developed to produce sampling matrix of any CS ratio with an ordered structure. Being equipped with the optimizationinspired recovery subnet guided by saliency information and a multi-block training scheme preventing blocking artifacts, CASNet jointly reconstructs the image blocks sampled at various sampling rates with one single model. To accelerate training convergence and improve network robustness, we propose an SVD-based initialization scheme and a random transformation enhancement (RTE) strategy, which are extensible without introducing extra parameters. All the CASNet components can be combined and learned end-to-end. We further provide a fourstage implementation for evaluation and practical deployments. Experiments demonstrate that CASNet outperforms other CS networks by a large margin, validating the collaboration and mutual supports among its components and strategies. Codes are available at https://github.com/Guaishou74851/CASNet.

Compressive Sensing in Image/Video Compression: Sampling, Coding, Reconstruction, and Codec Optimization

“…Another optimization aspect of the CS codec is to improve the adaptability while processing different images [85,86]. The key idea is to exploit the saliency information of images, and then allocate more sensing resources to these salient regions but fewer to nonsalient regions.…”

“…Figure 10 Given that image information is often unevenly distributed, an effective approach to enhance the restored image quality involves optimizing CS ratio allocations based on saliency distribution. The works of [86,87] define saliency as the locations exhibiting a low spatial correlation with their surroundings. As illustrated in Figure 10, the block enclosed in the crimson box should be assigned a higher CS ratio than the one within the light red.…”

“…This adjustment is justified by the former's intricate details and richer information content. Being equipped with the optimization-inspired recovery subnet guided by saliency information and a multi-block training scheme that prevents blocking artifacts in [86], this content-aware scalable network (CASNet) can jointly reconstruct the image blocks sampled at various CS ratios with one single model. The PSNR/SSIM and time-consuming results for various CS algorithms on Set11 [55] are shown in Table 2.…”

“…ReconNet [55] CSNet+ [56] AMP-Net [58] OPINE-Net [59] DeepBCS [60] DPA-Net [78] ISTA-Net++ [80] FSOINET [81] DPC-DUN [82] COAST [84] CASNet [86] MRVCS [90] VCSL [92] JVCSR [100]…”

An improved uncertainty autoencoder with blurred measurements

“…The trainable encoding and decoding network has enabled adaptively updating of the network, hence enjoying better reconstructing performance. The learning-based algorithms such as CASNet [3] and AMP-net [4] focus on the sparsity regularization in terms of the measurements, which train the encoder by minimizing the distance between the current estimation and the original output. Though the reconstruction performances have been improved, the measurements have not been particularly dealt with, resulting in poor harmony between the encoder and the decoder.…”