Frequency-Based Enhancement Network for Efficient Super-Resolution

Behjati, Parichehr; Rodríguez, Paul; Fernández, Carles; Mehri, Armin; Roca, Francesc; Ozawa, Seiichi; Gonzàlez, Jordi

doi:10.1109/access.2022.3176441

Cited by 7 publications

(3 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we compare our FSANet with state-of-the-art methods, including SRCNN [3], FSRCNN [9], VDSR [6], HDRN [48], CARN [49], MemNet [7], IMDN [50], LAPAR-A [51], SRMD [52], A2F-L [53], and FENet [54].…”

Section: Comparison and Visualization Of Psnr And Ssim Resultsmentioning

confidence: 99%

Frequency-Separated Attention Network for Image Super-Resolution

Qu,

Li,

Yao

2024

Applied Sciences

View full text Add to dashboard Cite

The use of deep convolutional neural networks has significantly improved the performance of super-resolution. Employing deeper networks to enhance the non-linear mapping capability from low-resolution (LR) to high-resolution (HR) images has inadvertently weakened the information flow and disrupted long-term memory. Moreover, overly deep networks are challenging to train, thus failing to exhibit the expressive capability commensurate with their depth. High-frequency and low-frequency features in images play different roles in image super-resolution. Networks based on CNNs, which should focus more on high-frequency features, treat these two types of features equally. This results in redundant computations when processing low-frequency features and causes complex and detailed parts of the reconstructed images to appear as smooth as the background. To maintain long-term memory and focus more on the restoration of image details in networks with strong representational capabilities, we propose the Frequency-Separated Attention Network (FSANet), where dense connections ensure the full utilization of multi-level features. In the Feature Extraction Module (FEM), the use of the Res ASPP Module expands the network’s receptive field without increasing its depth. To differentiate between high-frequency and low-frequency features within the network, we introduce the Feature-Separated Attention Block (FSAB). Furthermore, to enhance the quality of the restored images using heuristic features, we incorporate attention mechanisms into the Low-Frequency Attention Block (LFAB) and the High-Frequency Attention Block (HFAB) for processing low-frequency and high-frequency features, respectively. The proposed network outperforms the current state-of-the-art methods in tests on benchmark datasets.

show abstract

Section: Comparison and Visualization Of Psnr And Ssim Resultsmentioning

confidence: 99%

Frequency-Separated Attention Network for Image Super-Resolution

Qu,

Li,

Yao

2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Also, the feedback mechanism is exploited by Li, Zhen, et al [26] to refine the output of the network. Another mechanism is proposed in FENet by Behjati et al [27] using a frequency-based enhancement network. Luo et al [28] proposed a novel adversarial neural degradation (AND) model for blind image SR to generate a wide range of complex degradation effects that are highly non-linear.…”

Section: A Image Super-resolutionmentioning

confidence: 99%

A Novel Zero-Shot Real World Spatio-Temporal Super-Resolution (ZS-RW-STSR) Model for Video Super-Resolution

Shukla,

Upadhyay,

Sharma

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Super-resolution (SR) of the degraded and real low-resolution (LR) video remains a challenging problem despite the development of deep learning-based SR models. Most existing state-ofthe-art networks focus on getting high-resolution (HR) videos from the corresponding down-sampled LR video but fail in scenarios with noisy or degraded low-resolution video. In this article, a novel real-world "zero-shot" video spatio-temporal SR model, i.e., 3D-Deep Convolutional Auto-Encoder (3D-CAE) guided attention-based deep spatio-temporal back-projection network has been proposed. 3D-CAE is utilized for extracting noise-free features from real low-resolution video and used in the attention-based deep spatio-temporal back-projection network for clean, high-resolution video reconstruction. In the proposed framework, the denoising loss of low-resolution video with high-resolution video reconstruction loss is jointly used in an end-to-end manner with a zero-shot setting. Further, Meta-learning is used to initialize the weights of the proposed model to take advantage of learning on the external dataset with internal learning in a zero-shot environment. To maintain the temporal coherency, we have used the Motion Compensation Transformer (MCT) for motion estimation and the Sub-Pixel Motion Compensation (SPMC) layer for motion compensation. We have evaluated the performance of our proposed model on REDS and Vid4 Dataset. The PSNR value of our model is 25.13 dB for the RealVSR dataset, which is 0.72 dB more than the next-best performing model, EAVSR+. For MVSR4x, our model provides 24.61 db PSNR, 0.67 dB more than the EAVSR+ model. Experimental results demonstrate the effectiveness of the proposed framework on degraded and noisy real low-resolution video compared to the existing methods. Furthermore, an ablation study has been conducted to highlight the contribution of 3D-CAE and attention layer to the overall network performance.

show abstract

“…Previous work introduced attention mechanisms [16] [17] into super resolution reconstruction to simulate spatial positions, channels, or the interdependence between the two to solve this problem. Recent frequencybased methods [18] use more complex convolution operations to preserve high frequency information in the network. Although these methods achieve good reconstruction results to a certain extent, the attention-based method does not distinguish between low frequency and high frequency features, while the frequency-based method simply uses the residual branch as high frequency information.…”

Section: Introductionmentioning

confidence: 99%

Feature Preserving and Enhancing Network for Image Super-Resolution

Su,

Li,

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Single image super-resolution (SISR) with deep convolutional neural networks has recently attracted increasing attention due to its potentials to generate rich details. To obtain better fidelity and visual quality, most of existing methods are of heavy design with the depth of network. However, the lack of highfrequency information in the deep network and the limit of non-local context relations of pixels remain the core issue. In addition, there are many arbitrary-scale image super-resolution tasks, so it is a focus that provides an effective and efficient model for super resolution with arbitrary-scale. To this end, we propose a Feature Preserving and Enhancing Network (FPEN) based on implicit representation, which is aims to preserving high-frequency information in deep network and enhancing non-local contextual features of pixels to output more realistic images at arbitrary scales. In particular, our proposed High Frequency Preserving Block (HFPB) can divide the features into high-frequency components and low-frequency components, and allocate more operations to high-frequency components to ensure that high-frequency components can be preserved in the network. Since high-frequency information contains the details and texture of the image, it can restore the finer details of the image. Moreover, Pixel Continuity Attention (PiCA) module our proposed utilizes visual cues observed from pixels to adaptively recalibrate the pixel range that needs attention in the image to better achieve feature enhancement and generate smoother images. Extensive experiments conducted on benchmark SISR models and datasets show that Feature Preserving and Enhancing Network can be employed for various SISR tasks with arbitrary-scale to obtain the better visual quality than other state-of-the-art SR algorithms.INDEX TERMS Image super resolution with arbitrary-scale, high frequency preservingsingle methods, non-local attention methods, implicit neural representation.

show abstract

Frequency-Based Enhancement Network for Efficient Super-Resolution

Cited by 7 publications

References 70 publications

Frequency-Separated Attention Network for Image Super-Resolution

Frequency-Separated Attention Network for Image Super-Resolution

A Novel Zero-Shot Real World Spatio-Temporal Super-Resolution (ZS-RW-STSR) Model for Video Super-Resolution

Feature Preserving and Enhancing Network for Image Super-Resolution

Contact Info

Product

Resources

About