Incorporating image priors with deep convolutional neural networks for image super-resolution

Liang, Yudong; Wang, Jinjun; Zhou, Sanping; Gong, Yi; Zheng, Nanning

doi:10.1016/j.neucom.2016.02.046

Cited by 72 publications

(28 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another wok that exploits various image priors during the training phase of a deep CNN [50] is called SCRNN-Pr. One aspect of prior information focuses edge/texture restoration and the other concentrates on gradual upscaling via parallel structure recurrence.…”

Section: B State Of the Art Methods On Image Srmentioning

confidence: 99%

Multimedia super-resolution via deep learning: A survey

Hayat

2018

Digital Signal Processing

View full text Add to dashboard Cite

The recent phenomenal interest in convolutional neural networks (CNNs) must have made it inevitable for the super-resolution (SR) community to explore its potential. The response has been immense and in the last three years, since the advent of the pioneering work, there appeared too many works not to warrant a comprehensive survey. This paper surveys the SR literature in the context of deep learning. We focus on the three important aspects of multimedia -namely image, video and multi-dimensions, especially depth maps. In each case, first relevant benchmarks are introduced in the form of datasets and state of the art SR methods, excluding deep learning. Next is a detailed analysis of the individual works, each including a short description of the method and a critique of the results with special reference to the benchmarking done. This is followed by minimum overall benchmarking in the form of comparison on some common dataset, while relying on the results reported in various works. Fig. 1: Backpropagation (after [7]).acceptable level of convergence whereby the optimized parameters should ideally classify each subsequent test case correctly. The birth of Convolutional neural networks (CNN) or ConvNets can be traced back to 1988 [15] 1 wherein backpropagation was employed to train a NN to classify handwritten digits. Subsequent works by LeCun evolved into what was later known as LeNet5 [17]. After that there's virtual lull till late noughties [18] when GPUs were efficient enough to culminate in the work [19]. Since then a floodgate has opened and we hear of various architectures in the form of AlexNet [20], ZFNet [21], GoogLeNet [22] DenseNet [23] etc.; for a detailed overview one can consult [18], [24].The metamorphosis from fully connected NN to locally connected NN to CNN is illustrated in Fig. 2. As can be seen, rather than being fully connected, the CNN employs convolutions leading to local connections, where each local region of the input is connected to a neuron in the output. The input to a CNN is in the form of multiple arrays, such as a color image with three 2D arrays (length × width) in accordance to RGB or YCbCr channels. The number of channels is called depth and constitutes the 3rd D; note that more than three channels are not uncommon, e.g with hyperspectral images. A CNN is made up of Layers with each layer transforming an input 3D volume to an output 3D volume [11], typically, via four distinct operations [25], viz. convolution, a non-linear activation function (ReLU), pooling or sub-sampling and classification (fully connected Layer). A simplified CNN is illustrated in Fig. 3 2 . A CNN can be described as several convolution layers with nonlinear activation functions (e.g. ReLU or sigmoid) applied to each layer. Each convolution layer applies several (may be thousands) distinct filters 3 (also called feature maps) and combines their results. These filters are automatically learnt during the training part based on the task in hand, e.g. if the task is image classification the learning concerns, a...

show abstract

Section: B State Of the Art Methods On Image Srmentioning

confidence: 99%

Multimedia super-resolution via deep learning: A survey

Hayat

2018

Digital Signal Processing

View full text Add to dashboard Cite

show abstract

“…Benefiting from the powerful non-linear mapping, SRCNN (Dong et al, 2014(Dong et al, , 2016 improves the performance dramatically compared with the traditional SR methods. Since training SRCNN model usually takes a very long time before convergence, Liang et al (2016) introduce Sobel edge detection so as to capture gradient information to accelerate the training convergence.…”

Section: Overall Schemementioning

confidence: 99%

Single satellite imagery simultaneous super-resolution and colorization using multi-task deep neural networks

Liu

Han

et al. 2018

Journal of Visual Communication and Image Representation

View full text Add to dashboard Cite

Satellite imagery is a kind of typical remote sensing data, which holds preponderance in large area imaging and strong macro integrity. However, for most commercial space usages, such as virtual display of urban traffic flow, virtual interaction of environmental resources, one drawback of satellite imagery is its low spatial resolution, failing to provide the clear image details. Moreover, in recent years, synthesizing the color for grayscale satellite imagery or recovering the original color of camouflage sensitive regions becomes an urgent requirement for large spatial objects virtual reality interaction. In this work, unlike existing works which solve these two problems separately, we focus on achieving image super-resolution (SR) and image colorization synchronously. Based on multi-task learning, we provide a novel deep neural network model to fulfill single satellite imagery SR and colorization simultaneously. By feeding back the color feature representations into the SR network and jointly optimizing such two tasks, our deep model successfully achieves the mutual cooperation between imagery reconstruction and image colorization.

show abstract

“…However, SRCNN ignores the image prior, which is a significant component for image recovery. Liang et al [34] introduce Sobel edge prior so as to capture gradient information to accelerate the training convergence. In fact, the method does reduce the training time but the resultant reconstruction enhancement is limited.…”

Section: Related Workmentioning

confidence: 99%

Single image super-resolution using a deep encoder–decoder symmetrical network with iterative back projection

et al. 2018

View full text Add to dashboard Cite

Image super-resolution (SR) usually refers to reconstructing a high resolution (HR) image from a low resolution (LR) image without losing high frequency details or reducing the image quality. Recently, image SR based on convolutional neural network (SRCNN) was proposed and has received much attention due to its end-to-end mapping simplicity and superior performance. This method, however, only using three convolution layers to learn the mapping from LR to HR, usually converges slowly and leads to the size of output image reducing significantly. To address these issues, in this work, we propose a novel deep encoder-decoder symmetrical neural network (DEDSN) for single image SR. This deep network is fully composed of symmetrical multiple layers of convolution and deconvolution and there is no pooling (down-sampling and upsampling) operations in the whole network so that image details degradation occurred in traditional convolutional frameworks is prevented. Additionally, in view of the success of the iterative back projection (IBP) algorithm in image SR, we further combine DEDSN with IBP network realization in this work. The new DEDSN-IBP model introduces the down sampling version of the ground truth image and calculates the simulation error as the prior guidance. Experimental results on benchmark data sets demonstrate that the proposed DEDSN model can achieve better performance than SRCNN and the improved DEDSN-IBP outperforms the reported state-of-the-art methods.

show abstract

Incorporating image priors with deep convolutional neural networks for image super-resolution

Cited by 72 publications

References 17 publications

Multimedia super-resolution via deep learning: A survey

Multimedia super-resolution via deep learning: A survey

Single satellite imagery simultaneous super-resolution and colorization using multi-task deep neural networks

Single image super-resolution using a deep encoder–decoder symmetrical network with iterative back projection

Contact Info

Product

Resources

About