Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural network

Lanaras, Charis; Bioucas-Dias, José M.; Galliani, Silvano; Baltsavias, E.; Schindler, Konrad

doi:10.1016/j.isprsjprs.2018.09.018

Cited by 262 publications

(214 citation statements)

References 45 publications

Supporting

Mentioning

213

Contrasting

Order By: Relevance

“…Nevertheless we included the results of this method for comparison. Under the assumption that source and guide images are available in large quantities we follow a common procedure from the literature [26,15]: under the assumption that the upsampling model is to some degree scale-invariant, one can downsample the available M × M data by the factor D to obtain synthetic training data for ×D upsampling. The model thus trained for upsampling (M/D) 2 → M 2 is then, at test time, applied to the actual super-resolution task M 2 → N 2 .…”

Section: Evaluation Settingsmentioning

confidence: 99%

Guided Super-Resolution As Pixel-to-Pixel Transformation

Lutio¹,

D’Aronco²,

Wegner

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

Guided super-resolution is a unifying framework for several computer vision tasks where the inputs are a lowresolution source image of some target quantity (e.g., perspective depth acquired with a time-of-flight camera) and a high-resolution guide image from a different domain (e.g., a grey-scale image from a conventional camera); and the target output is a high-resolution version of the source (in our example, a high-res depth map). The standard way of looking at this problem is to formulate it as a super-resolution task, i.e., the source image is upsampled to the target resolution, while transferring the missing high-frequency details from the guide. Here, we propose to turn that interpretation on its head and instead see it as a pixel-to-pixel mapping of the guide image to the domain of the source image. The pixel-wise mapping is parametrised as a multi-layer perceptron, whose weights are learned by minimising the discrepancies between the source image and the downsampled target image. Importantly, our formulation makes it possible to regularise only the mapping function, while avoiding regularisation of the outputs; thus producing crisp, naturallooking images. The proposed method is unsupervised, using only the specific source and guide images to fit the mapping. We evaluate our method on two different tasks, superresolution of depth maps and of tree height maps. In both cases we clearly outperform recent baselines in quantitative comparisons, while delivering visually much sharper outputs.

show abstract

Section: Evaluation Settingsmentioning

confidence: 99%

Guided Super-Resolution As Pixel-to-Pixel Transformation

Lutio¹,

D’Aronco²,

Wegner

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Greater spatial resolution allows for a finer analysis and hence, more knowledge about the true condition of the earth. Previous works have been mainly focused on obtaining all 13 bands in 10m resolution using both the information of lower resolution bands and the existing 10m resolution bands (Lanaras et al, 2018, Gargiulo et al, 2018. However, these methods cannot be used for further increasing the resolution of RGB and NIR bands (e.g., 5m or 2.5m) as they require having bands at the target resolution.…”

Section: Introductionmentioning

confidence: 99%

Super-Resolution for Sentinel-2 Images

Galar

Sesma

Ayala

et al. 2019

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Obtaining Sentinel-2 imagery of higher spatial resolution than the native bands while ensuring that output imagery preserves the original radiometry has become a key issue since the deployment of Sentinel-2 satellites. Several studies have been carried out on the upsampling of 20m and 60m Sentinel-2 bands to 10 meters resolution taking advantage of 10m bands. However, how to super-resolve 10m bands to higher resolutions is still an open problem. Recently, deep learning-based techniques has become a de facto standard for single-image super-resolution. The problem is that neural network learning for super-resolution requires image pairs at both the original resolution (10m in Sentinel-2) and the target resolution (e.g., 5m or 2.5m). Since there is no way to obtain higher resolution images for Sentinel-2, we propose to consider images from others sensors having the greatest similarity in terms of spectral bands, which will be appropriately pre-processed. These images, together with Sentinel-2 images, will form our training set. We carry out several experiments using state-of-the-art Convolutional Neural Networks for single-image super-resolution showing that this methodology is a first step toward greater spatial resolution of Sentinel-2 images.

show abstract

“…where g is a pan-sharpening CNN with filter parameters θ. The conventional methods [16], [19] use the L2 loss as…”

Section: B Proposed S3 Lossmentioning

confidence: 99%

S3: A Spectral-Spatial Structure Loss for Pan-Sharpening Networks

Choi

Kim

2020

IEEE Geosci. Remote Sensing Lett.

View full text Add to dashboard Cite

Recently, many deep-learning-based pan-sharpening methods have been proposed for generating high-quality pansharpened (PS) satellite images. These methods focused on various types of convolutional neural network (CNN) structures, which were trained by simply minimizing a spectral loss between network outputs and the corresponding high-resolution multispectral (MS) target images. However, due to different sensor characteristics and acquisition times, high-resolution panchromatic (PAN) and low-resolution MS image pairs tend to have large pixel misalignments, especially for moving objects in the images. Conventional CNNs trained with only the spectral loss with these satellite image datasets often produce PS images of low visual quality including double-edge artifacts along strong edges and ghosting artifacts on moving objects. In this letter, we propose a novel loss function, called a spectral-spatial structure (S3) loss, based on the correlation maps between MS targets and PAN inputs. Our proposed S3 loss can be very effectively utilized for pan-sharpening with various types of CNN structures, resulting in significant visual improvements on PS images with suppressed artifacts.

show abstract

Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural network

Cited by 262 publications

References 45 publications

Guided Super-Resolution As Pixel-to-Pixel Transformation

Guided Super-Resolution As Pixel-to-Pixel Transformation

Super-Resolution for Sentinel-2 Images

S3: A Spectral-Spatial Structure Loss for Pan-Sharpening Networks

Contact Info

Product

Resources

About