Multi-Image Blind Super-Resolution of 3D Scenes

Punnappurath, Abhijith; Nimisha, T M; Rajagopalan, A. N.

doi:10.1109/tip.2017.2723243

Cited by 4 publications

(5 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, the possibility of many similarities along with the imaging blur, and noise often makes the problem ill-posed. Different prior information help in address the ill-posed nature of the problem [10,11,12,13,14,15,16,17,18].…”

Section: Conventional Single Image Srmentioning

confidence: 99%

Deep Networks for Image and Video Super-Resolution

Purohit¹,

Mandal²,

Rajagopalan³

2022

Preprint

View full text Add to dashboard Cite

Efficiency of gradient propagation in intermediate layers of convolutional neural networks is of key importance for superresolution task. To this end, we propose a deep architecture for single image super-resolution (SISR), which is built using efficient convolutional units we refer to as mixed-dense connection blocks (MDCB). The design of MDCB combines the strengths of both residual and dense connection strategies, while overcoming their limitations. To enable super-resolution for multiple factors, we propose a scale-recurrent framework which reutilizes the filters learnt for lower scale factors recursively for higher factors. This leads to improved performance and promotes parametric efficiency for higher factors. We train two versions of our network to enhance complementary image qualities using different loss configurations. We further employ our network for video super-resolution task, where our network learns to aggregate information from multiple frames and maintain spatio-temporal consistency. The proposed networks lead to qualitative and quantitative improvements over state-of-the-art techniques on image and video super-resolution benchmarks. IntroductionSingle image super-resolution (SISR) aims to estimate a high-resolution (HR) image from a low-resolution (LR) input image, and is an ill-posed problem. Due to its diverse applicability starting from surveillance to medical diagnosis, and from remote sensing to HDTV, the SISR problem has gathered substantial attention from computer vision and image processing community. The ill-posed nature of the problem is generally addressed by learning a LR-HR mapping function in a constrained environment using example HR-LR patch pairs.One way is to learn a mapping function that linearly correlates the HR-LR patch pairs. Such linear functions can be easily learned with few example images as has been practiced by some SR approaches [1,2,3]. But, linear mapping between such patch pairs may not be representative enough to learn different complex structures present in the image. The mapping function would benefit from learning non-linear relationships between HR-LR patch pairs. Recent convolutional neural network (CNN) based models are quite efficient for such a purpose, and can be useful in extracting relevant features by making deeper models. However, deeper models often face vanishing/exploding gradient issues, which can be partially mitigated by using residual mapping [4,5]. Deep residual models has been employed for higher level vision tasks, where batch normalization is generally used for a useful class-specific normalized representation. However, such representation is not much useful in low-level vision task such as SR [6]. Most deep CNN based SR models do not make full use of the hierarchical features from the original LR images. Thus, the scope of improvement in performance is there in effective employment of the hierarchical features from all the convolutional layers, as has been employed by a residual dense network [7] using a sequence of residual dense block...

show abstract

Section: Conventional Single Image Srmentioning

confidence: 99%

Deep Networks for Image and Video Super-Resolution

Purohit¹,

Mandal²,

Rajagopalan³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Since there could be many similarities, one needs to regularize the problem. Thus, most of the conventional approaches focus on discovering regularization techniques in SR such as Tikhinov [49], total-variation [28], Markov random field [17], non-local-mean [26,11,27], sparsity-based prior [46,47,10], and so on [9,27,36,37,39,3,2,34,33,43,35].…”

Section: Related Workmentioning

confidence: 99%

Image Superresolution using Scale-Recurrent Dense Network

Purohit¹,

Mandal²,

Rajagopalan³

2022

Preprint

View full text Add to dashboard Cite

Recent advances in the design of convolutional neural network (CNN) have yielded significant improvements in the performance of image superresolution (SR). The boost in performance can be attributed to the presence of residual or dense connections within the intermediate layers of these networks. The efficient combination of such connections can reduce the number of parameters drastically while maintaining the restoration quality. In this paper, we propose a scale recurrent SR architecture built upon units containing series of dense connections within a residual block (Residual Dense Blocks (RDBs)) that allow extraction of abundant local features from the image. Our scale recurrent design delivers competitive performance for higher scale factors while being parametrically more efficient as compared to current state-of-the-art approaches. To further improve the performance of our network, we employ multiple residual connections in intermediate layers (referred to as Multi-Residual Dense Blocks), which improves gradient propagation in existing layers. Recent works have discovered that conventional loss functions can guide a network to produce results which have high PSNRs but are perceptually inferior. We mitigate this issue by utilizing a Generative Adversarial Network (GAN) based framework and deep feature (VGG) losses to train our network. We experimentally demonstrate that different weighted combinations of the VGG loss and the adversarial loss enable our network outputs to traverse along the perception-distortion curve. The proposed networks perform favorably against existing methods, both perceptually and objectively (PSNR-based) with fewer parameters.

show abstract

“…Pour cette raison, dans la suite de ce manuscrit, nous nommerons cet IMF le modèle WarpHR. Dans la littérature SR de nombreuses variantes de cet IMF sont considérées, intégrant divers dégradations supplémentaires : flou de bougé [Punnappurath et al, 2017], variations d'illumination ou d'exposition au sein de la séquence BR [Capel, 2001], dégradations dues à la compression d'images [Gunturk et al, 2002] ...…”

Section: Modèle "Warphr"unclassified

“…La référence [Punnappurath et al, 2017] s'intéresse à la SR sur des images BR polluées par du flou de bougé. La géométrie 3D de la scène est intégrée à l'IMF sous forme de palliers de profondeur.…”

Section: Les Méthodes Généralistesunclassified