Landsat Super-Resolution Enhancement Using Convolution Neural Networks and Sentinel-2 for Training

Pouliot, Darren; Latifovic, Rasim; Pasher, Jon; Duffe, Jason

doi:10.3390/rs10030394

Cited by 83 publications

(52 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The rise of deep learning has also advanced single-image super-resolution (Lim et al, 2017;Kim et al, 2016). Moreover, such super-resolution has been applied to Sentinel-2 and Landsat-8 images (Pouliot et al, 2018). All these works have in common that they predict images of higher spatial resolution, meaning that what is learned is a generic prior on the local structure of highresolution images; whereas our method increases resolution of particular bands in a more informed and more accurate manner, by transferring the texture from available high-resolution bands; effectively learning a prior on the correlations across the spectrum (or, equivalently, on the high-resolution structure of some bands conditioned on the known high-resolution structure of other bands).…”

Section: Related Workmentioning

confidence: 99%

Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural network

Lanaras

Bioucas-Dias

Galliani

et al. 2018

ISPRS Journal of Photogrammetry and Remote Sensing

260

180

View full text Add to dashboard Cite

The Sentinel-2 satellite mission delivers multi-spectral imagery with 13 spectral bands, acquired at three different spatial resolutions. The aim of this research is to super-resolve the lower-resolution (20 m and 60 m Ground Sampling Distance -GSD) bands to 10 m GSD, so as to obtain a complete data cube at the maximal sensor resolution. We employ a stateof-the-art convolutional neural network (CNN) to perform end-to-end upsampling, which is trained with data at lower resolution, i.e., from 40→20 m, respectively 360→60 m GSD. In this way, one has access to a virtually infinite amount of training data, by downsampling real Sentinel-2 images. We use data sampled globally over a wide range of geographical locations, to obtain a network that generalises across different climate zones and land-cover types, and can super-resolve arbitrary Sentinel-2 images without the need of retraining. In quantitative evaluations (at lower scale, where ground truth is available), our network, which we call DSen2, outperforms the best competing approach by almost 50% in RMSE, while better preserving the spectral characteristics. It also delivers visually convincing results at the full 10 m GSD.

show abstract

Section: Related Workmentioning

confidence: 99%

Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural network

Lanaras

Bioucas-Dias

Galliani

et al. 2018

ISPRS Journal of Photogrammetry and Remote Sensing

260

180

View full text Add to dashboard Cite

show abstract

“…In another example, [22] use deep neural networks for simultaneous 4× super-resolution and colorization of satellite imagery. Several papers [41,25,35,20,28] modify or leverage SRCNN [7] and/or VDSR [15] to successfully super-resolve Jilin-1, SPOT, Pleiades, Sentinel-2, and Landsat imagery.…”

Section: Super-resolution Techniques and Application To Overhead Imagerymentioning

confidence: 99%

The Effects of Super-Resolution on Object Detection Performance in Satellite Imagery

Shermeyer

Etten

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

190

View full text Add to dashboard Cite

We explore the application of super-resolution techniques to satellite imagery, and the effects of these techniques on object detection algorithm performance. Specifically, we enhance satellite imagery beyond its native resolution, and test if we can identify various types of vehicles, planes, and boats with greater accuracy than native resolution. Using the Very Deep Super-Resolution (VDSR) framework and a custom Random Forest Super-Resolution (RFSR) framework we generate enhancement levels of 2×, 4×, and 8× over five distinct resolutions ranging from 30 cm to 4.8 meters. Using both native and super-resolved data, we then train several custom detection models using the SIMRDWN object detection framework. SIMRDWN combines a number of popular object detection algorithms (e.g. SSD, YOLO) into a unified framework that is designed to rapidly detect objects in large satellite images. This approach allows us to quantify the effects of super-resolution techniques on object detection performance across multiple classes and resolutions. We also quantify the performance of object detection as a function of native resolution and object pixel size. For our test set we note that performance degrades from mean average precision (mAP) = 0.53 at 30 cm resolution, down to mAP = 0.11 at 4.8 m resolution. Super-resolving native 30 cm imagery to 15 cm yields the greatest benefit; a 13 − 36% improvement in mAP. Superresolution is less beneficial at coarser resolutions, though still provides a small improvement in performance.

show abstract

“…CNN architecture has become a hot topic in the field of deep learning [27]. The CNN architecture provides a new method for remote sensing image high-level feature extraction.…”

Section: Proposed Convolutional Neural Network Architecturementioning

confidence: 99%

Multilevel Cloud Detection for High-Resolution Remote Sensing Imagery Using Multiple Convolutional Neural Networks

Yang

Fan

Bilal

et al. 2018

IJGI

View full text Add to dashboard Cite

In high-resolution image data, multilevel cloud detection is a key task for remote sensing data processing. Generally, it is difficult to obtain high accuracy for multilevel cloud detection when using satellite imagery which only contains visible and near-infrared spectral bands. So, multilevel cloud detection for high-resolution remote sensing imagery is challenging. In this paper, a new multilevel cloud detection technique is proposed based on the multiple convolutional neural networks for high-resolution remote sensing imagery. In order to avoid input the entire image into the network for cloud detection, the adaptive simple linear iterative clustering (A-SCLI) algorithm was applied to the segmentation of the satellite image to obtain good-quality superpixels. After that, a new multiple convolutional neural networks (MCNNs) architecture is designed to extract multiscale features from each superpixel, and the superpixels are marked as thin cloud, thick cloud, cloud shadow, and non-cloud. The results suggest that the proposed method can detect multilevel clouds and obtain a high accuracy for high-resolution remote sensing imagery.

show abstract

Landsat Super-Resolution Enhancement Using Convolution Neural Networks and Sentinel-2 for Training

Cited by 83 publications

References 29 publications

Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural network

Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural network

The Effects of Super-Resolution on Object Detection Performance in Satellite Imagery

Multilevel Cloud Detection for High-Resolution Remote Sensing Imagery Using Multiple Convolutional Neural Networks

Contact Info

Product

Resources

About