Exploring Sparsity in Image Super-Resolution for Efficient Inference

Wang, Longguang; Dong, Xiaoyu; Wang, Yingqian; Ying, Xinyi; Lin, Zaiping; An, Wei; Guo, Yulan

doi:10.1109/cvpr46437.2021.00488

Cited by 212 publications

(63 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To further test the adequacy of our proposed GFSR model, we compare it with seven CNN-based SR methods: SRCNN [30], VDSR [6], LapSRN [21], IDN [25], DPSR [22], IMDN [26], PAN [35], LAPAR-A [27], SMSR [28], DASR [24] and DeFiAN [23]. Inspired by [57], a self-ensemble strategy is introduced to further improve the proposed GFSR, and the improved model is named as GFSR+.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

“…In addition, to decrease the computational cost, Li et al [27] propose a linearly-assembled pixel-adaptive regression network (LAPAR), which transforms the conventional learning of LR to HR mappings into a multiple predefined regression task on linear filter coefficients. Wang et al [28] proposed a sparse mask SR model (SMSR) to boost the inference efficiency by exploring the sparsity in SR tasks. SMSR determines significant and redundant regions by spatial mask learning and channel mask learning, respectively, and maintains superior performance while skipping redundant computations.…”

Section: Cnn-based Sr Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Gradient-Guided and Multi-Scale Feature Network for Image Super-Resolution

et al. 2022

View full text Add to dashboard Cite

Recently, deep-learning-based image super-resolution methods have made remarkable progress. However, most of these methods do not fully exploit the structural feature of the input image, as well as the intermediate features from the intermediate layers, which hinders the ability of detail recovery. To deal with this issue, we propose a gradient-guided and multi-scale feature network for image super-resolution (GFSR). Specifically, a dual-branch structure network is proposed, including the trunk branch and the gradient one, where the latter is used to extract the gradient feature map as structural prior to guide the image reconstruction process. Then, to absorb features from different layers, two effective multi-scale feature extraction modules, namely residual of residual inception block (RRIB) and residual of residual receptive field block (RRRFB), are proposed and embedded in different network layers. In our RRIB and RRRFB structures, an adaptive weighted residual feature fusion block (RFFB) is investigated to fuse the intermediate features to generate more beneficial representations, and an adaptive channel attention block (ACAB) is introduced to effectively explore the dependencies between channel features to further boost the feature representation capacity. Experimental results on several benchmark datasets demonstrate that our method achieves superior performance against state-of-the-art methods in terms of both subjective visual quality and objective quantitative metrics.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Section: Cnn-based Sr Modelsmentioning

confidence: 99%

Gradient-Guided and Multi-Scale Feature Network for Image Super-Resolution

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In order to prove effectiveness of this network, we compare MBMFN with other advanced lightweight super-resolution reconstruction algorithms with up-sampling factors of ×2, ×3 and ×4. Include SRCNN [4], VDSR [5], DRCN [24], MemNet [25], CARN [26], IMDN [11], DNCL [27], FilterNeL [28], RFDN [12], CFSRCNN [29], SeaNet-baseline [30], SMSR [31], MSFIN [13], Cross-SRN [32], MRFN [33], MADNet-L F [34]. The experimental results are shown in Table 4.…”

Section: Comparison With Other Advanced Algorithmsmentioning

confidence: 99%

Image Reconstruction of Multibranch Feature Multiplexing Fusion Network with Mixed Multilayer Attention

et al. 2022

View full text Add to dashboard Cite

Image super-resolution reconstruction achieves better results than traditional methods with the help of the powerful nonlinear representation ability of convolution neural network. However, some existing algorithms also have some problems, such as insufficient utilization of phased features, ignoring the importance of early phased feature fusion to improve network performance, and the inability of the network to pay more attention to high-frequency information in the reconstruction process. To solve these problems, we propose a multibranch feature multiplexing fusion network with mixed multilayer attention (MBMFN), which realizes the multiple utilization of features and the multistage fusion of different levels of features. To further improve the network’s performance, we propose a lightweight enhanced residual channel attention (LERCA), which can not only effectively avoid the loss of channel information but also make the network pay more attention to the key channel information and benefit from it. Finally, the attention mechanism is introduced into the reconstruction process to strengthen the restoration of edge texture and other details. A large number of experiments on several benchmark sets show that, compared with other advanced reconstruction algorithms, our algorithm produces highly competitive objective indicators and restores more image detail texture information.

show abstract

“…Mask generation. As several work has illustrated the effectiveness of treating network sparsity as a probabilistic event [7,19], the importance vector can be either a learnable parameter itself, or a Bernoulli sampling from a 2 × C learnable parameter with reparameterization trick [20]. We denote these two methods as Deterministic and Stochastic.…”

Section: Ablation Studymentioning

confidence: 99%

Exploring Structural Sparsity in Neural Image Compression

Yin¹,

Li²,

Wen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Neural image compression have reached or out-performed traditional methods (such as JPEG, BPG, WebP). However, their sophisticated network structures with cascaded convolution layers bring heavy computational burden for practical deployment. In this paper, we explore the structural sparsity in neural image compression network to obtain real-time acceleration without any specialized hardware design or algorithm. We propose a simple plug-in adaptive binary channel masking(ABCM) to judge the importance of each convolution channel and introduce sparsity during training. During inference, the unimportant channels are pruned to obtain slimmer network and less computation. We implement our method into three neural image compression networks with different entropy models to verify its effectiveness and generalization, the experiment results show that up to 7× computation reduction and 3× acceleration can be achieved with negligible performance drop.

show abstract

Exploring Sparsity in Image Super-Resolution for Efficient Inference

Cited by 212 publications

References 30 publications

Gradient-Guided and Multi-Scale Feature Network for Image Super-Resolution

Gradient-Guided and Multi-Scale Feature Network for Image Super-Resolution

Image Reconstruction of Multibranch Feature Multiplexing Fusion Network with Mixed Multilayer Attention

Exploring Structural Sparsity in Neural Image Compression

Contact Info

Product

Resources

About