Ensemble Super-Resolution With a Reference Dataset

Abstract

By developing sophisticated image priors or designing deep(er) architectures, a variety of image Super-Resolution (SR) approaches have been proposed recently and achieved very promising performance. A natural question that arises is whether these methods can be reformulated into a unifying framework and whether this framework assists in SR reconstruction? In this paper, we present a simple but effective single image SR method based on ensemble learning, which can produce a better performance than that could be obtained from any of SR methods to be ensembled (or called component superresolvers). Based on the assumption that better component superresolver should have larger ensemble weight when performing SR reconstruction, we present a Maximum A Posteriori (MAP) estimation framework for the inference of optimal ensemble weights. Specially, we introduce a reference dataset, which is composed of High-Resolution (HR) and Low-Resolution (LR) image pairs, to measure the super-resolution abilities (prior knowledge) of different component super-resolvers. To obtain the optimal ensemble weights, we propose to incorporate the reconstruction constraint, which states that the degenerated HR image should be equal to the LR observation one, as well as the prior knowledge of ensemble weights into the MAP estimation framework. Moreover, the proposed optimization problem can be solved by an analytical solution. We study the performance of the proposed method by comparing with different competitive approaches, including four state-of-the-art non-deep learning based methods, four latest deep learning based methods and one ensemble learning based method, and prove its effectiveness and superiority on three public datasets.

Head and neck tumor segmentation convolutional neural network robust to missing PET/CT modalities using channel dropout

“…Ensemble modeling is a technique in which predictions from multiple individual models are synthesized to yield a single higher performing model (Jiang et al 2020 , Sudharson and Kokil 2020 ). This allows a single model to leverage distinct model attributes, such as those of dilated (improved capture of global information) and traditional (improved capture of fine details) convolutions.…”

A Lightweight Dense Connected Approach with Attention on Single Image Super-Resolution

“…Dong et al unprecedentedly introduced a three-layer CNN framework into the SISR and proposed a super-resolution convolutional neural network (SRCNN) [18], which exhibited a remarkable performance compared to the traditional works [8][9][10][11][12][13][14][15][16][17] and opened the way for neural network-based SR research. After that, plenty of approaches based on convolution neural networks were proposed.…”

“…Traditional SISR methods can be divided into three categories: reconstruction-based methods [8,9], interpolation methods [10][11][12], and learning-based methods [13][14][15][16][17]. Recently, with the rapid development of neural networks, Convolutional Neural Network-based (CNN-based) SR methods have achieved remarkable performances [18][19][20][21][22][23][24].…”

Cascading and Enhanced Residual Networks for Accurate Single-Image Super-Resolution

“…Dong et al [7], [8] first proposed a convolutional model to solve the SISR problem in 2014, which became a milestone in the image restoration area. Since then, more complicated networks were designed to enhance the performance [10], [16], [18], [19], [21], [23], [25], [39], [40], [58]. Lim et al [25] proposed a very deep and wide model with residual blocks and achieved satisfactory performance in terms of both peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) [48].…”