Multimedia super-resolution via deep learning: A survey

Hayat, Khizar

doi:10.1016/j.dsp.2018.07.005

Cited by 82 publications

(33 citation statements)

References 157 publications

(264 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The "super-resolution" problem of converting low-resolution images to higher resolution has been attempted with machine learning [4][5][6][7][8][9][10][11][12]. The super-resolution problem is an ill-posed inverse problem, with many high-resolution images possible for a given low-resolution image.…”

Section: Introductionmentioning

confidence: 99%

Optimal physical preprocessing for example-based super-resolution

Robey¹,

Ganapati²

2018

Opt. Express

View full text Add to dashboard Cite

In example-based super-resolution, the function relating low-resolution images to their high-resolution counterparts is learned from a given dataset. This data-driven approach to solving the inverse problem of increasing image resolution has been implemented with deep learning algorithms. In this work, we explore modifying the imaging hardware in order to collect more informative low-resolution images for better ultimate high-resolution image reconstruction. We show that this "physical preprocessing" allows for improved image reconstruction with deep learning in Fourier ptychographic microscopy.Fourier ptychographic microscopy is a technique allowing for both high resolution and high field-of-view at the cost of temporal resolution. In Fourier ptychographic microscopy, variable illumination patterns are used to collect multiple low-resolution images. These low-resolution images are then computationally combined to create an image with resolution exceeding that of any single image from the microscope. We use deep learning to jointly optimize the illumination pattern with the post-processing reconstruction algorithm for a given sample type, allowing for single-shot imaging with both high resolution and high field-of-view. We demonstrate, with simulated data, that the joint optimization yields improved image reconstruction as compared with sole optimization of the post-processing reconstruction algorithm.

show abstract

Section: Introductionmentioning

confidence: 99%

Optimal physical preprocessing for example-based super-resolution

Robey¹,

Ganapati²

2018

Opt. Express

View full text Add to dashboard Cite

show abstract

“…In recent years, deep learning and artificial intelligence have been widely used in various industries, especially in the field of computer vision, and have achieved better results than traditional methods [14,15]. Using the feed-forward depth network methods of CNN is the mainstream of the current super-resolution reconstruction field after sparse representation.…”

Section: Related Workmentioning

confidence: 99%

Image Super-Resolution Based on CNN Using Multilabel Gene Expression Programming

et al. 2020

View full text Add to dashboard Cite

Current mainstream super-resolution algorithms based on deep learning use a deep convolution neural network (CNN) framework to realize end-to-end learning from low-resolution (LR) image to high-resolution (HR) images, and have achieved good image restoration effects. However, as the number of layers in the network is increased, better results are not necessarily obtained, and there will be problems such as slow training convergence, mismatched sample blocks, and unstable image restoration results. We propose a preclassified deep-learning algorithm (MGEP-SRCNN) using Multilabel Gene Expression Programming (MGEP), which screens out a sample sub-bank with high relevance to the target image before image block extraction, preclassifies samples in a multilabel framework, and then performs nonlinear mapping and image reconstruction. The algorithm is verified through standard images, and better objective image quality is obtained. The restoration effect under different magnification conditions is also better.The gradient descent method continuously adjusts the weights to obtain a mapping from LR to HR. Its simple network structure design and excellent image restoration results created a precedent for deep learning in super-resolution image reconstruction. However, the SRCNN method does not obtain better results when the number of layers in the network is increased. The training convergence rate is slow and it is not suitable for multiscale amplification.Based on the above problems, we propose an improved deep learning algorithm for Gene Expression Programming (GEP) preclassification. This method uses the GEP multilabeling algorithm to classify the trained image set and select a subset of samples that are related in color and texture feature categories, thereby reducing the complexity of the convolutional neural network parameters. We compare the performance of our approach to that of other state-of-the-art methods and obtain improved objective image quality. Related WorkIn recent years, deep learning and artificial intelligence have been widely used in various industries, especially in the field of computer vision, and have achieved better results than traditional methods [14,15]. Using the feed-forward depth network methods of CNN is the mainstream of the current super-resolution reconstruction field after sparse representation. Such methods focus less on the reconstruction speed and more on whether the high-resolution map can be better restored at a large magnification. They have better generalization ability and ability to characterize the high-level characteristics, compared with traditional shallow learning algorithms. SRCNN/Fast SRCNN (FSRCNN)Dong et al. [5] first proposed the use of a convolutional neural network for image super-resolution reconstruction. An LR image was first enlarged to the target size using Bicubic interpolation, and then a nonlinear mapping was performed through a three-layer convolutional network. The obtained results were output as high-resolution images, and good results were obtained. As s...

show abstract

“…Various deep learning methods have been applied in the past, to solve the SISR problem, many of which have been summarized in [23]. First, Dong et al proposed in [9] the replacement of all steps to produce a high resolution imagefeature extraction then mapping then reconstruction -by a single neural network.…”

Section: Single Image Super Resolutionmentioning

confidence: 99%

Efficient Single Image Super Resolution Using Enhanced Learned Group Convolutions

Jain

Bansal

Singh

et al. 2018

Neural Information Processing

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs) have demonstrated great results for the single-image super-resolution (SISR) problem. Currently, most CNN algorithms promote deep and computationally expensive models to solve SISR. However, we propose a novel SISR method that uses relatively less number of computations. On training, we get group convolutions that have unused connections removed. We have refined this system specifically for the task at hand by removing unnecessary modules from original CondenseNet. Further, a reconstruction network consisting of deconvolutional layers has been used in order to upscale to high resolution. All these steps significantly reduce the number of computations required at testing time. Along with this, bicubic upsampled input is added to the network output for easier learning. Our model is named SRCondenseNet. We evaluate the method using various benchmark datasets and show that it performs favourably against the state-of-the-art methods in terms of both accuracy and number of computations required.

show abstract

Multimedia super-resolution via deep learning: A survey

Cited by 82 publications

References 157 publications

Optimal physical preprocessing for example-based super-resolution

Optimal physical preprocessing for example-based super-resolution

Image Super-Resolution Based on CNN Using Multilabel Gene Expression Programming

Efficient Single Image Super Resolution Using Enhanced Learned Group Convolutions

Contact Info

Product

Resources

About