MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution

Lu, Liying; Li, Wenbo; Xin, Tao; Lu, Jiangbo; Jia, Jiaya

doi:10.1109/cvpr46437.2021.00630

Cited by 113 publications

(90 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare our method against extensive existing super-resolution methods including [6, 9-11, 18, 22-24, 26, 33, 34, 36, 38, 40, 41]. For [22,38], the T1WI is regarded as the reference image. We implement three variants of our method with different model sizes, including: 1) For 'Ours-S', 𝑑 is set to 16, two feature extraction and enhancement stages are used, and each RRDB block only contains two RDBs composed of three convolutions; 2) For 'Our-M', 𝑑 is set to 16, three feature extraction and enhancement stages are used, and each RRDB block only contains three RDBs composed of three convolutions; 3) 'Our-L' is the final variant of our method with default settings.…”

Section: Comparison With Other Methodsmentioning

confidence: 99%

“…Inspired by the rapid development in natural image super-resolution [7,16,26,37,41], CNN models have been the mainstream solutions for the MR image superresolution [4,28,42]. According to the current studies [22,23,38], the low-frequency signals in MR data are relatively easy to reconstruct. In contrast, super-resolving the high-frequency signals, such as structures and textures, remains the main challenge.…”

Section: Related Work 21 Mr Image Super-resolutionmentioning

confidence: 99%

“…Considering that there would be a large domain gap between the gradient fields of different modality data, directly injecting features of the T1 modality into those of the T2 modality would not be the optimal solution. Inspired from [13,22], we adopt a point-wise adaptive instance normalization scheme to reduce the cross-modality representation gap as shown in Fig. 5.…”

Section: Inter-modality Attentionmentioning

confidence: 99%

“…The adaptive normalization in [22] calculates global affine parameters are calculated in each batch normalization layer. Our devised normalization to differs from it in the adoption of point-wise affine parameters, which model local distribution deviation more finely.…”

Section: Inter-modality Attentionmentioning

confidence: 99%

See 3 more Smart Citations

Cross-Modality High-Frequency Transformer for MR Image Super-Resolution

Fang¹,

Zhang²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Section: Comparison With Other Methodsmentioning

confidence: 99%

Section: Related Work 21 Mr Image Super-resolutionmentioning

confidence: 99%

Section: Inter-modality Attentionmentioning

confidence: 99%

Section: Inter-modality Attentionmentioning

confidence: 99%

See 2 more Smart Citations

Cross-Modality High-Frequency Transformer for MR Image Super-Resolution

Fang¹,

Zhang²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

“…Image super-resolution is an important image restoration task in computer vision [8,21,23,[29][30][31]. Since SRCNN [5] first applied early convolutional neural networks to solve SR tasks.…”

Section: Image Super-resolutionmentioning

confidence: 99%

What Hinders Perceptual Quality of PSNR-oriented Methods?

Xu¹,

Mi²,

Zheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

PSNR-oriented models (POMs) have achieved great success and are widely adopted in many image superresolution (SR) applications. However, details of images generated by these models are usually over-smoothed. Many existing methods improve the perceptual quality of generated images by designing models with larger sizes to optimize the models' learning ability, or using GAN-based models to generate fake details. Unfortunately, those methods cost too much labor effort to be practical in real-world applications. To solve this issue, in this paper, we discover two factors that inhibit POMs from achieving high perceptual quality: 1) center-oriented optimization (COO) problem and 2) model's low-frequency tendency. First, POMs tend to generate an SR image whose position in the feature space is closest to the distribution center of all potential high-resolution (HR) images, resulting in such POMs losing high-frequency details. Second, 90% area of an image consists of low-frequency signals; in contrast, human perception relies on an image's high-frequency details. However, POMs apply the same calculation to process differentfrequency areas, so that POMs tend to restore the lowfrequency regions. Based on these two factors, we propose a Detail Enhanced Contrastive Loss (DECLoss), by combining a high-frequency enhancement module and spatial contrastive learning module, to reduce the influence of the COO problem and low-Frequency tendency. Experimental results show the efficiency and effectiveness when applying DECLoss on several regular SR models. E.g, in EDSR, our proposed method achieves 3.60× faster learning speed compared to a GAN-based method with a subtle degradation in visual quality. In addition, our final results show that an SR network equipped with our DECLoss generates more realistic and visually pleasing textures compared to state-of-the-art methods.

show abstract

Face hallucination using multisource references and cross‐scale dual residual fusion mechanism

Wang

Jian

et al. 2022

Int J of Intelligent Sys

View full text Add to dashboard Cite

There is an increasing interest in enhancing the quality of low-resolution (LR) facial images for various social life applications. Existing methods often use domainspecific prior knowledge, which is effective in improving the face super-resolution model's performance.However, it is challenging to obtain rich and accurate prior information from LR inputs in real-world scenarios, which can limit the robustness and generalization ability of the developed face super-resolution model. In this paper, a multisource reference-based face super-resolution Network, namely MSRNet, is proposed. Without considering the prior knowledge of faces, the network can reconstruct a LR face image with a magnitude factor of 8 under the guidance of multiple reference face images of different identities.By constructing an "appearance-alike" reference data set Face_Ref, the designed MSRNet aims to fully exploit the local and spatially similar high frequency information between the distinct references and the current face. More specifically, to effectively combine the information from multiple references, a cross-scale and cross-space feature fusion mechanism is introduced for external and internal references, and then the enhanced local semantics are finally incorporated into the high-resolution face reconstruction. The

show abstract

MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution

Cited by 113 publications

References 21 publications

Cross-Modality High-Frequency Transformer for MR Image Super-Resolution

Cross-Modality High-Frequency Transformer for MR Image Super-Resolution

What Hinders Perceptual Quality of PSNR-oriented Methods?

Face hallucination using multisource references and cross‐scale dual residual fusion mechanism

Contact Info

Product

Resources

About