Limits of Learning-Based Superresolution Algorithms

Learning-based superresolution (SR) are popular SR techniques that use application dependent priors to infer the missing details in low resolution images (LRIs). However, their performance still deteriorates quickly when the magnification factor is moderately large. This leads us to an important problem: "Do limits of learning-based SR algorithms exist?" In this paper, we attempt to shed some light on this problem when the SR algorithms are designed for general natural images (GNIs). We first define an expected risk for the SR algorithms that is based on the root mean squared error between the superresolved images and the ground truth images. Then utilizing the statistics of GNIs, we derive a closed form estimate of the lower bound of the expected risk. The lower bound can be computed by sampling real images. By computing the curve of the lower bound w.r.t. the magnification factor, we can estimate the limits of learning-based SR algorithms, at which the lower bound of expected risk exceeds a relatively large threshold. We also investigate the sufficient number of samples to guarantee an accurate estimation of the lower bound. Learning-Based SR AlgorithmsLearning-based SR algorithms are new SR techniques that may have started from the seminal papers by Freeman Low Res. Image High Res.

show abstract

“…In subsections 2.3 and 2.6, we will provide the ideas of proving the above two theorems. The full details of proof can be found in [9].…”

Section: Resultsmentioning

confidence: 99%

Limits of Learning-Based Superresolution Algorithms

Lin

Tang

et al. 2007

2007 IEEE 11th International Conference on Computer Vision

Self Cite

View full text Add to dashboard Cite

show abstract

“…Traditional image processing algorithms mainly rely on basic digital image processing techniques. Generally, there are three categories: interpolation-based algorithms [21][22][23], degenerate-model-based algorithms [24][25][26] and learning-based algorithms [27][28][29][30].…”

Section: Traditional Methodsmentioning

confidence: 99%

Super-Resolution Reconstruction of Cytoskeleton Image Based on A-Net Deep Learning Network

Chen

Bai²,

Che³

et al. 2022

Micromachines

View full text Add to dashboard Cite

To date, live-cell imaging at the nanometer scale remains challenging. Even though super-resolution microscopy methods have enabled visualization of sub-cellular structures below the optical resolution limit, the spatial resolution is still far from enough for the structural reconstruction of biomolecules in vivo (i.e., ~24 nm thickness of microtubule fiber). In this study, a deep learning network named A-net was developed and shows that the resolution of cytoskeleton images captured by a confocal microscope can be significantly improved by combining the A-net deep learning network with the DWDC algorithm based on a degradation model. Utilizing the DWDC algorithm to construct new datasets and taking advantage of A-net neural network’s features (i.e., considerably fewer layers and relatively small dataset), the noise and flocculent structures which originally interfere with the cellular structure in the raw image are significantly removed, with the spatial resolution improved by a factor of 10. The investigation shows a universal approach for exacting structural details of biomolecules, cells and organs from low-resolution images.

show abstract

“…From another perspective, for the natural images, the reconstruction error of any learning‐based SR algorithms is bounded by Lin et al . [40]

\begin{matrix} right leftthickmathspace.5em \\ ξ \cdot {()(, trace ()(, false(I - U V false) {boldΨ}_{h} false(I - U V false)^{normalT}) + {∥(bold-italicI - bold-italicU V) \overset{false¯}{bold-italich}∥}^{2})}^{1 / 2} \end{matrix}

where ξ is a constant related with the magnification factor and signal dimension, U and V denote the up‐sampling matrix and decimation matrix, respectively, and Ψ h and

\bar{h}

are the covariance matrix and the mean of HR patches h . From (13), we can observe the following two points.…”

Section: Sr Methods Based On Perceptual Criteriamentioning

confidence: 99%

Single‐frame image super‐resolution inspired by perceptual criteria

Zhou

Liao

2015

IET Image Processing

View full text Add to dashboard Cite

In this study, the authors consider the problem of image super-resolution (SR) in terms of the perceptual criteria. Existing SR methods treat the traditional mean-squared error (MSE) as an irreplaceable objective function. However, MSE has been widely criticised since it is inconsistent with visual perception of human beings. The perceptual criteria, including the structural similarity (SSIM) index and feature similarity (FSIM) index, have been reported to be more effective in assessing image quality. Therefore SSIM and FSIM are included for the SR task in this study. Specifically, the authors first propose to reform principal component analysis (PCA), which is named as visual perceptual PCA (VP-PCA), by adopting SSIM as the object function. Subsequently, to accomplish the SR task, the authors cluster the training data and perform VP-PCA on each cluster to calculate the coefficients. Finally, based on the principle of FSIM, the traditional SR results and the SR results using VP-PCA are combined to form our fused results. Experimental results are provided to show the superiority of the proposed method over several state-of-the-art methods in both quantitative and visual comparisons.

show abstract

Limits of Learning-Based Superresolution Algorithms

Cited by 27 publications

References 31 publications

Limits of Learning-Based Superresolution Algorithms

Limits of Learning-Based Superresolution Algorithms

Super-Resolution Reconstruction of Cytoskeleton Image Based on A-Net Deep Learning Network

Single‐frame image super‐resolution inspired by perceptual criteria

Contact Info

Product

Resources

About