Deep Hyperspectral Prior: Single-Image Denoising, Inpainting, Super-Resolution

Sidorov, Oleksii; Hardeberg, Jon Yngve

doi:10.1109/iccvw.2019.00477

Cited by 167 publications

(135 citation statements)

References 38 publications

Supporting

Mentioning

135

Contrasting

Order By: Relevance

“…In this method, we utilize bicubic interpolation to upsample the original LR MS image, but this method may introduce extra faulty information and degrade the fusion accuracy. For future work, we can adopt deep image prior [51][52] or single-image SR methods [53][54] to finish the process of upsample, in order to further reduce spectral and spatial distortions before fusion. Another line of research concerns designing an attention fusion network that can adopt channel or spatial attention to reduce the redundant information during feature fusion.…”

Section: Resultsmentioning

confidence: 99%

Pansharpening via Unsupervised Convolutional Neural Networks

Luo

Zhou

Feng

et al. 2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Pansharpening is normally utilized to take full advantage of all the available spectral and spatial information that are derived from a low-spatial-resolution (LR) multispectral (MS) image and its associated high-spatial-resolution (HR) panchromatic (PAN) image, respectively, producing a fused MS image with high spectral and spatial resolutions. Many methods have been recently developed based on convolutional neural networks (CNNs) for the pansharpening task, but most of them still have some drawbacks: 1) The information cannot efficiently flow in their simple stacked convolutional architectures, thereby hindering the representation ability of the networks; 2) They are commonly trained using supervised learning, which does not only require an extra effort to produce the simulated training data, but can also lead to scale-related problems in the fusion results. In this paper, we propose a novel unsupervised CNN-based pansharpening method to overcome these limitations. Specifically, we design an iterative network architecture, in which a PANguided strategy and a set of skip connections (SC) are adopted to continuously extract and fuse the features from the input, thus enhancing the information reuse and transmission. Besides, we propose a new loss function for unsupervised training in which the relationships between the input MS and PAN images and the fused MS image are used to design the spatial constrains and spectral consistency, respectively. The typical quality index with noreference (QNR) is also added to this function to further adjust the spectral and spatial qualities. The designed loss function allows the network to be learned only on input images, without any handcrafted labels (reference HR MS image). We evaluated the effectiveness of our designed network architecture and the combined loss function, and the experiments testify that our unsupervised strategy can also obtain promising results with minor spectral and spatial distortions compared with other traditional and supervised methods.

show abstract

Section: Resultsmentioning

confidence: 99%

Pansharpening via Unsupervised Convolutional Neural Networks

Luo

Zhou

Feng

et al. 2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Zhu et al [30] propose patch-based inpainting method for forensics images. Using the same technique of encoder-decoder network, Sidorov and Hardeberg [31] proposed an architecture for denoising, inpainting and super-resolution for noised, inpainted and low-resolution images, respectively. Zeng et al [32] built a pyramidal-context architecture called PEN-NET for high-quality image inpainting.…”

Section: Cnn-based Approachesmentioning

confidence: 99%

“…Table 2 summarizes CNN-based method with a description of the type of data used for image inpainting. [29] Blind CNN Grayscale Sidorov et al 2019 [31] 3D CNN RGB Zeng et al 2019 [32] Pyramid-context encoder network RGB Pathak et al 2016 [34] Context-encoder, CNN RGB Sasaki et al 2017 [35] CNN line drawing images Nakamura et al 2017 [37] CNN Image with text Xiang et al 2017 [39] CNN, GAN Damaged old pictures Cai et al 2018 [41] CNN RGB…”

Section: Cnn-based Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Image Inpainting: A Review

Elharrouss

Almaadeed

Al‐Maadeed

et al. 2019

Neural Process Lett

264

100

View full text Add to dashboard Cite

Although image inpainting, or the art of repairing the old and deteriorated images, has been around for many years, it has gained even more popularity because of the recent development in image processing techniques. With the improvement of image processing tools and the flexibility of digital image editing, automatic image inpainting has found important applications in computer vision and has also become an important and challenging topic of research in image processing. This paper is a brief review of the existing image inpainting approaches we first present a global vision on the existing methods for image inpainting. We attempt to collect most of the existing approaches and classify them into three categories, namely, sequential-based, CNN-based and GAN-based methods. In addition, for each category, a list of methods for the different types of distortion on the images is presented. Furthermore, collect a list of the available datasets and discuss these in our paper. This is a contribution for digital image inpainting researchers trying to look for the available datasets because there is a lack of datasets available for image inpainting. As the final step in this overview, we present the results of real evaluations of the three categories of image inpainting methods performed on the datasets used, for the different types of image distortion. In the end, we also present the evaluations metrics and discuss the performance of these methods in terms of these metrics. This overview can be used as a reference for image inpainting researchers, and it can also facilitate the comparison of the methods as well as the datasets used. The main contribution of this paper is the presentation of the three categories of image inpainting methods along with a list of available datasets that the researchers can use to evaluate their proposed methodology against.

show abstract

“…[ 17 , 18 ]. In recent years, the deep-learning (DL) technique has become a research hotspot in various fields, such as object classification and segmentation [ 19 , 20 ], super-resolution [ 21 , 22 ], image denoising [ 23 , 24 ], medical image reconstruction [ 25 , 26 ], etc. In addition to the above applications, it is also adopted in radar signal-processing applications.…”

Section: Introductionmentioning

confidence: 99%

Millimeter-Wave Array Radar-Based Human Gait Recognition Using Multi-Channel Three-Dimensional Convolutional Neural Network

Jiang

Zhang

Yang

et al. 2020

Sensors

View full text Add to dashboard Cite

At present, there are two obvious problems in radar-based gait recognition. First, the traditional radar frequency band is difficult to meet the requirements of fine identification with due to its low carrier frequency and limited micro-Doppler resolution. Another significant problem is that radar signal processing is relatively complex, and the existing signal processing algorithms are poor in real-time usability, robustness and universality. This paper focuses on the two basic problems of human gait detection with radar and proposes a human gait classification and recognition method based on millimeter-wave array radar. Based on deep-learning technology, a multi-channel three-dimensional convolution neural network is proposed on the basis of improving the residual network, which completes the classification and recognition of human gait through the hierarchical extraction and fusion of multi-dimensional features. Taking the three-dimensional coordinates, motion speed and intensity of strong scattering points in the process of target motion as network inputs, multi-channel convolution is used to extract motion features, and the classification and recognition of typical daily actions are completed. The experimental results show that we have more than 92.5% recognition accuracy for common gait categories such as jogging and normal walking.

show abstract

Deep Hyperspectral Prior: Single-Image Denoising, Inpainting, Super-Resolution

Cited by 167 publications

References 38 publications

Pansharpening via Unsupervised Convolutional Neural Networks

Pansharpening via Unsupervised Convolutional Neural Networks

Image Inpainting: A Review

Millimeter-Wave Array Radar-Based Human Gait Recognition Using Multi-Channel Three-Dimensional Convolutional Neural Network

Contact Info

Product

Resources

About