Cloud Removal in Satellite Images Using Spatiotemporal Generative Networks

Sarukkai, Vishnu; Jain, Anirudh; Uzkent, Burak; Ermon, Stefano

doi:10.1109/wacv45572.2020.9093564

Cited by 82 publications

(112 citation statements)

References 25 publications

Supporting

Mentioning

112

Contrasting

Order By: Relevance

“…Unlike the model of [12], our architecture follows that of cycle-consistent GAN and has the advantage of not requiring pixelwise correspondences between cloudy and noncloudy optical training data, thereby also allowing for training or fine-tuning on data where such a requirement may not be met. Complementary to the SAR-optical data fusion approach to cloud removal, recent contributions proposed integrating information of repeated observations over time [10], [11]. The work indicates promising results but trades temporal resolution for obtaining a single cloud-free observation, whereas our approach predicts one cloud-free output per cloudy input image and, thus, allows for sequence-to-sequence translation.…”

Section: A Related Workmentioning

confidence: 98%

“…The work indicates promising results but trades temporal resolution for obtaining a single cloud-free observation, whereas our approach predicts one cloud-free output per cloudy input image and, thus, allows for sequence-to-sequence translation. Moreover, current multitemporal approaches make strong assumptions about the maximum permissible amount of cloud-coverage affecting individual images in the input time series, which is required to be no more than 25% or 50% of cloud coverage for the method of [10] and 10%-30% in the work of [11]. Our curated data sets evidence that such strict requirements on the percentage of cloudiness may, oftentimes, not be met in practice.…”

Section: A Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Multisensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagery

Ebel

Meraner

Schmitt

et al. 2021

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

The majority of optical observations acquired via spaceborne Earth imagery are affected by clouds. While there is numerous prior work on reconstructing cloud-covered information, previous studies are, oftentimes, confined to narrowly defined regions of interest, raising the question of whether an approach can generalize to a diverse set of observations acquired at variable cloud coverage or in different regions and seasons. We target the challenge of generalization by curating a large novel data set for training new cloud removal approaches and evaluate two recently proposed performance metrics of image quality and diversity. Our data set is the first publically available to contain a global sample of coregistered radar and optical observations, cloudy and cloud-free. Based on the observation that cloud coverage varies widely between clear skies and absolute coverage, we propose a novel model that can deal with either extreme and evaluate its performance on our proposed data set. Finally, we demonstrate the superiority of training models on real over synthetic data, underlining the need for a carefully curated data set of real observations. To facilitate future research, our data set is made available online.

show abstract

Section: A Related Workmentioning

confidence: 98%

Section: A Related Workmentioning

confidence: 99%

Multisensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagery

Ebel

Meraner

Schmitt

et al. 2021

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Sarukkai et al. [SJUE19] cast the problem of cloud removal as a conditional image synthesis challenge, and propose a trainable spatio‐temporal generator network to remove clouds. Their model is trained and validated on a new large‐scale spatio‐temporal dataset constituted by real images.…”

Section: Related Workmentioning

confidence: 99%

Thin Cloud Removal for Single RGB Aerial Image

Song

Xiao

Zhang

et al. 2021

Computer Graphics Forum

View full text Add to dashboard Cite

Acquired above variable clouds, aerial images contain the components of ground reflection and cloud effect. Due to the non‐uniformity, clouds in aerial images are even harder to remove than haze in terrestrial images. This paper proposes a divide‐and‐conquer scheme to remove the thin translucent clouds in a single RGB aerial image. Based on colour attenuation prior, we design a kind of veiling metric that indicates the local concentration of clouds effectively. By this metric, an aerial image containing thickness‐varied clouds is segmented into multiple regions. Each region is veiled by clouds of nearly‐equal concentration, and hence subject to common assumptions, such as boundary constraint on transmission. The atmospheric light in each region is estimated by the modified local colour‐line model and composed into a spatially‐varying airlight map for the entire image. Then scene transmission is estimated and further refined by a weighted L1‐norm based contextual regularization. Finally, we recover ground reflection via the atmospheric scattering model. We verify our cloud removal method on a number of aerial images containing thin clouds and compare our results with classical single‐image dehazing methods and the state‐of‐the‐art learning‐based declouding method, respectively.

show abstract

“…However, per-pixel annotations of clouds are time consuming and tedious, while per-image annotated datasets only require one click for the whole image. In this paper, we present a cloud detector that has learned from a per-image annotated dataset of panchromatic satellite images, derived from the single-image dataset [18] appearing in Figure 1. The approach is generic and can be extended to any sort of single-band image.…”

Section: Introductionmentioning

confidence: 99%

A CNN Cloud Detector for Panchromatic Satellite Images

Rodríguez

Anger

Franchis

et al. 2021

2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS

View full text Add to dashboard Cite

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

show abstract

Cloud Removal in Satellite Images Using Spatiotemporal Generative Networks

Cited by 82 publications

References 25 publications

Multisensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagery

Multisensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagery

Thin Cloud Removal for Single RGB Aerial Image

A CNN Cloud Detector for Panchromatic Satellite Images

Contact Info

Product

Resources

About