IRGUN : Improved Residue Based Gradual Up-Scaling Network for Single Image Super Resolution

Sharma, Manoj Kumar; Mukhopadhyay, Rudrabha; Upadhyay, Avinash; Koundinya, Sriharsha; Shukla, Ankit; Chaudhury, Santanu

doi:10.1109/cvprw.2018.00128

Cited by 6 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent advancements in image super-resolution (SR) have been driven by deep learning techniques like [5], [7], [8], [16]- [21]. While effective, these methods rely on knowing the exact degradation kernel, posing limitations in real-world applications.…”

Section: A Image Super-resolutionmentioning

confidence: 99%

See 1 more Smart Citation

A Novel Zero-Shot Real World Spatio-Temporal Super-Resolution (ZS-RW-STSR) Model for Video Super-Resolution

Shukla,

Upadhyay,

Sharma

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Super-resolution (SR) of the degraded and real low-resolution (LR) video remains a challenging problem despite the development of deep learning-based SR models. Most existing state-ofthe-art networks focus on getting high-resolution (HR) videos from the corresponding down-sampled LR video but fail in scenarios with noisy or degraded low-resolution video. In this article, a novel real-world "zero-shot" video spatio-temporal SR model, i.e., 3D-Deep Convolutional Auto-Encoder (3D-CAE) guided attention-based deep spatio-temporal back-projection network has been proposed. 3D-CAE is utilized for extracting noise-free features from real low-resolution video and used in the attention-based deep spatio-temporal back-projection network for clean, high-resolution video reconstruction. In the proposed framework, the denoising loss of low-resolution video with high-resolution video reconstruction loss is jointly used in an end-to-end manner with a zero-shot setting. Further, Meta-learning is used to initialize the weights of the proposed model to take advantage of learning on the external dataset with internal learning in a zero-shot environment. To maintain the temporal coherency, we have used the Motion Compensation Transformer (MCT) for motion estimation and the Sub-Pixel Motion Compensation (SPMC) layer for motion compensation. We have evaluated the performance of our proposed model on REDS and Vid4 Dataset. The PSNR value of our model is 25.13 dB for the RealVSR dataset, which is 0.72 dB more than the next-best performing model, EAVSR+. For MVSR4x, our model provides 24.61 db PSNR, 0.67 dB more than the EAVSR+ model. Experimental results demonstrate the effectiveness of the proposed framework on degraded and noisy real low-resolution video compared to the existing methods. Furthermore, an ablation study has been conducted to highlight the contribution of 3D-CAE and attention layer to the overall network performance.

show abstract

Section: A Image Super-resolutionmentioning

confidence: 99%

“…Though various techniques have evolved for single image super-resolution (SISR) [5]- [8], the VSR is still a demanding and ill-posed problem. Contrary to SISR, where a single image is super-resolved, VSR intends to encapsulate the interframe alignments while performing frame-to-frame superresolution.…”

Section: Introductionmentioning

confidence: 99%

A Novel Zero-Shot Real World Spatio-Temporal Super-Resolution (ZS-RW-STSR) Model for Video Super-Resolution

Shukla,

Upadhyay,

Sharma

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Medical image synthesis is a broad field set at the intersection of medical imaging, traditional machine learning approaches, and newer, deep learning algorithms. Synthesis applications address artifact correction or dataset completion, 1,2 superresolution, [3][4][5][6] modality prediction, [7][8][9] imaging dose reduction, 10,11 or dataset augmentation. 12,13 Realistic and accurate synthetic images can have a large clinical impact, notably to limit patient radiation exposure, eliminate redundant imaging procedures, or avoid potentially harmful contrast agent injections.…”

Section: Introductionmentioning

confidence: 99%

On the proper use of structural similarity for the robust evaluation of medical image synthesis models

2022

View full text Add to dashboard Cite

To propose good practices for using the structural similarity metric (SSIM) and reporting its value. SSIM is one of the most popular image quality metrics in use in the medical image synthesis community because of its alleged superiority over voxel-by-voxel measurements like the average error or the peak signal noise ratio (PSNR). It has seen massive adoption since its introduction, but its limitations are often overlooked. Notably, SSIM is designed to work on a strictly positive intensity scale, which is generally not the case in medical imaging. Common intensity scales such as the Houndsfield units (HU) contain negative numbers, and they can also be introduced by image normalization techniques such as the z-normalization. Methods: We created a series of experiments to quantify the impact of negative values in the SSIM computation. Specifically, we trained a three-dimensional (3D) U-Net to synthesize T2-weighted MRI from T1-weighted MRI using the BRATS 2018 dataset. SSIM was computed on the synthetic images with a shifted dynamic range. Next, to evaluate the suitability of SSIM as a loss function on images with negative values, it was used as a loss function to synthesize znormalized images. Finally, the difference between two-dimensional (2D) SSIM and 3D SSIM was investigated using multiple 2D U-Nets trained on different planes of the images. Results:The impact of the misuse of the SSIM was quantified; it was established that it introduces a large downward bias in the computed SSIM. It also introduces a small random error that can change the relative ranking of models. The exact values for this bias and error depend on the quality and the intensity histogram of the synthetic images. Although small, the reported error is significant considering the small SSIM difference between state-of -the-art models. It was shown therefore that SSIM cannot be used as a loss function when images contain negative values due to major errors in the gradient calculation, resulting in under-performing models. 2D SSIM was also found to be overestimated in 2D image synthesis models when computed along the plane of synthesis, due to the discontinuities between slices that is typical of 2D synthesis methods. Conclusion: Various types of misuse of the SSIM were identified, and their impact was quantified. Based on the findings, this paper proposes good practices when using SSIM, such as reporting the average over the volume of the image containing tissue and appropriately defining the dynamic range.

show abstract

Gradually Growing Residual and Self-attention Based Dense Deep Back Projection Network for Large Scale Super-Resolution of Image

Sharma

Upadhyay

Singh

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

IRGUN : Improved Residue Based Gradual Up-Scaling Network for Single Image Super Resolution

Cited by 6 publications

References 24 publications

A Novel Zero-Shot Real World Spatio-Temporal Super-Resolution (ZS-RW-STSR) Model for Video Super-Resolution

A Novel Zero-Shot Real World Spatio-Temporal Super-Resolution (ZS-RW-STSR) Model for Video Super-Resolution

On the proper use of structural similarity for the robust evaluation of medical image synthesis models

Gradually Growing Residual and Self-attention Based Dense Deep Back Projection Network for Large Scale Super-Resolution of Image

Contact Info

Product

Resources

About