Super-resolution of PROBA-V images using convolutional neural networks

Abstract

ESA's PROBA-V Earth observation satellite enables us to monitor our planet at a large scale, studying the interaction between vegetation and climate and provides guidance for important decisions on our common global future. However, the interval at which high resolution images are recorded spans over several days, in contrast to the availability of lower resolution images which is often daily. We collect an extensive dataset of both, high and low resolution images taken by PROBA-V instruments during monthly periods to investigate Multi Image Super-resolution, a technique to merge several low resolution images to one image of higher quality. We propose a convolutional neural network that is able to cope with changes in illumination, cloud coverage and landscape features which are challenges introduced by the fact that the different images are taken over successive satellite passages over the same region. Given a bicubic upscaling of low resolution images taken under optimal conditions, we find the Peak Signal to Noise Ratio of the reconstructed image of the network to be higher for a large majority of different scenes. This shows that applied machine learning has the potential to enhance large amounts of previously collected earth observation data during multiple satellite passes.

Multi-Image Super Resolution of Remotely Sensed Images Using Residual Attention Deep Neural Networks

“…To evaluate the obtained results, we need to use a slightly modified version of PSNR and SSIM [62] criteria to take into consideration all the aspects we considered in the previous section to obtain a proper loss function. Martens et al [58] propose a corrected version of the PSNR, called cPSNS, that is obtained from a corrected mean squared error (cMSE). The computation of the cMSE is performed in the same way as we did for the loss in Eq.…”

“…To train our model, we exploit the dataset released by the Advanced Concept Team of the European Space Agency (ESA) [58]. This dataset has been specifically conceived for MISR problems, and it is composed of several images taken by the Proba-V satellite 3 in the two different spectral bands RED and NIR (near-infrared).…”

Satellite Image Multi-Frame Super-Resolution Using 3D Wide-Activation Neural Networks

“…In this work we used the set of images from the vegetation observation satellite PROBA-V of the European Space Agency (ESA) [18] provided in the context of the ESA's super resolution competition PROBA-V, which took place between 01.11.2018 and 31.05.2019 [20].…”

“…As described in Märtens et al [18], the data-set used for both training and testing is composed as follows:…”

“…In this technical note we present a technique that takes as a strong baseline Kim et al [16] 3DSRnet framework, but adapted for satellite image MFSR and replacing 3DCNN blocks with wide activation blocks [17]. This method core is a 3D wide activation residual network that was fully trained and tested on the PROBA-V dataset [18] on a low-specifications home laptop computer with only 4GB of GPU memory. Despite being low on resources and based on a simple architecture, this method topped Kelvin's ESA challenge in February 2020 [19].…”