Learning Super-Resolution for Sentinel-2 Images With Real Ground Truth Data From a Reference Satellite

Abstract

Abstract. Copernicus program via its Sentinel missions is making earth observation more accessible and affordable for everybody. Sentinel-2 images provide multi-spectral information every 5 days for each location. However, the maximum spatial resolution of its bands is 10m for RGB and near-infrared bands. Increasing the spatial resolution of Sentinel-2 images without additional costs, would make any posterior analysis more accurate. Most approaches on super-resolution for Sentinel-2 have focused on obtaining 10m resolution images for those at lower resolutions (20m and 60m), taking advantage of the information provided by bands of finer resolutions (10m). Otherwise, our focus is on increasing the resolution of the 10m bands, that is, super-resolving 10m bands to 2.5m resolution, where no additional information is available. This problem is known as single-image super-resolution and deep learning-based approaches have become the state-of-the-art for this problem on standard images. Obviously, models learned for standard images do not translate well to satellite images. Hence, the problem is how to train a deep learning model for super-resolving Sentinel-2 images when no ground truth exist (Sentinel-2 images at 2.5m). We propose a methodology for learning Convolutional Neural Networks for Sentinel-2 image super-resolution making use of images from other sensors having a high similarity with Sentinel-2 in terms of spectral bands, but greater spatial resolution. Our proposal is tested with a state-of-the-art neural network showing that it can be useful for learning to increase the spatial resolution of RGB and near-infrared bands of Sentinel-2.

SEG-ESRGAN: A Multi-Task Network for Super-Resolution and Semantic Segmentation of Remote Sensing Images

“…As mentioned before, Sentinel-2 satellites are of great importance for the RS community, with many authors addressing the SR of the 20 and 60 m bands [31][32][33] but few authors tackling the SR of the 10 m bands. For instance, Galar et al [34] proposed the use of PlanetScope (2.5 m) in combination with Sentinel-2 bands, Panagiotopoulou et al [35] introduced the use of SPOT-7 imagery (2.5 m), while other authors used WorldView imagery to work with Sentinel-2 as well [7,36].…”

MuS2: A Benchmark for Sentinel-2 Multi-Image Super-Resolution

“…Recently, several real-life SISR datasets have been elaborated which encompass pairs of original LR and HR images [6]. Some of the SISR techniques applied to S-2 were validated with PlanetScope [3] and WorldView-3 [7] images treated as a real HR reference, but these data have not been released for the community to serve as a benchmark. Preparing such datasets for MISR is much more costly and troublesome, as multiple LR images are necessary in this case.…”

“…Our benchmark dataset is composed of images acquired by S-2 and WV-2 satellites. The S-2 images were downloaded from the Copernicus Open Access Hub, and we accessed the WV-2 images published within the European Cities dataset 3 . The S-2 data are organized into tiles, each of which contains a 100 × 100 km MSI, and aggregated into products that are distributed at different processing levels.…”

“…2 The data preparation source code is available at https://codeocean.com/ capsule/8131193/tree/v1. 3 The European Cities dataset is freely available at https://earth.esa.int/web/ guest/-/worldview-2-european-cities-dataset. correction.…”

“…This poses a challenge when preparing the data for training and validation. Recent advances in satellite image SR include SISR [3] and MISR [4] techniques developed to overcome the problem of insufficient spatial resolution of Sentinel-2 (S-2) multispectral images (MSIs). They are composed of 13 bands, whose resolution ranges from 60 m ground sampling distance (GSD) to 10 m GSD [5].…”