SEN12MS-CR-TS: A Remote-Sensing Data Set for Multimodal Multitemporal Cloud Removal

Ebel, Patrick; Xu, Yajin; Schmitt, Michael; Zhu, Xiao Xiang

doi:10.1109/tgrs.2022.3146246

Cited by 78 publications

(59 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our future work will focus on designing more robust architecture for the proposed data setting. We will furthermore investigate the combination with image reconstruction methods for spatio-temporal cloud removal [61]. We will also like to extend the method for fine grained changed objection detection [62], [63].…”

Section: Discussionmentioning

confidence: 99%

Supervised Change Detection Using Prechange Optical-SAR and Postchange SAR Data

Saha

Shahzad

Ebel

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

Self Cite

View full text Add to dashboard Cite

Change detection using satellite/aerial images is used to quantify the impacts of many natural and man-made disasters. At the occurrence of such events, both pre-change optical and Synthetic Aperture Radar (SAR) images can be obtained by going back in time. However, the availability of the post-change optical image is often hindered by the presence of artifacts like clouds.To circumnavigate this, we propose a novel change detection data setting which uses both optical and SAR images prechange, yet only SAR imagery post-change. For this challenging scenario, we propose a Siamese network that processes the pre-change and post-change SAR inputs using a shared set of weights, while the pre-change optical input is processed using a network that do not share the weights with the SAR inputs. The encoded weights from the three networks are fused and finally decoded using a common decoder to obtain the change map. Our model effectively fuses multi-sensor information and can obtain satisfactory result despite the absence of the post-change optical image. Experimental results on a multi-sensor urban dataset demonstrate the effectiveness of the proposed approach. Code is available at https:

show abstract

Section: Discussionmentioning

confidence: 99%

Supervised Change Detection Using Prechange Optical-SAR and Postchange SAR Data

Saha

Shahzad

Ebel

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

Self Cite

View full text Add to dashboard Cite

show abstract

“…on the data set of [35]. Moreover, recent publications have provided novel large-scale data sets for cloud detection or removal in time-series [12], [13], [36], which may serve for an extended version of our analysis. With respect to the cloud detector algorithm, s2cloudless was chosen for being commonly deployed, easily applicable and performing well [37], [38].…”

Section: ) Outliers As Distractorsmentioning

confidence: 99%

“…While clouds are characterised in great detail [6], [7] and different approaches for handling them have been investigated, less effort has been spent to investigate what exactly its effects on remote sensing applications are. The existing approaches range from learning cloud removal for preprocessing [8], [9], [10], [11], [12], [13] to familiarizing neural networks with clouds by including cloud-covered observations in the training data set, such that the models learn to ignore clouds irrelevant to the task at hand [14], [4], [3]. Such approaches that include cloudy images in the training process are limited to samples with transparent clouds or samples where the crucial features for a classification are not covered.…”

Section: Introductionmentioning

confidence: 99%

Explaining the Effects of Clouds on Remote Sensing Scene Classification

Gawlikowski

Ebel

Schmitt

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Most of Earth is covered by haze or clouds, impeding the constant monitoring of our planet. Preceding works have documented the detrimental effects of cloud coverage on remote sensing applications and proposed ways to approach this issue. However, up-to-now, little effort has been spent on understanding how exactly atmospheric disturbances impede the application of modern machine learning methods to Earth observation data. Specifically, we consider the effects of haze and cloud coverage on a scene classification task. We provide a thorough investigation of how classifiers trained on cloud-free data fail once they encounter noisy imagery -a common scenario encountered when deploying pretrained models for remote sensing to real use cases. We show how and why remote sensing scene classification suffers from cloud coverage. Based on a multi-stage analysis, including explainability approaches applied to the predictions, we work out four different types of effects that clouds have on the scene prediction. The contribution of our work is to deepen the understanding of the effects of clouds on common remote sensing applications, and consequently guide the development of more robust methods.

show abstract

“…However, as the thickness of clouds increases, all the land signals in the optical bands are obstructed. Consequently, multitemporalbased approaches have been proposed to restore the missing information with data from other time periods (Zhang et al, 2021;Shen et al, 2019;Ebel et al, 2022). However, when encountering continual cloudy days, cloud-free reference data from an adjacent period is largely unavailable.…”

Section: Related Workmentioning

confidence: 99%

Exploring the Potential of SAR Data for Cloud Removal in Optical Satellite Imagery

Fang¹,

Shi²,

Ebel³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The challenge of the cloud removal task can be alleviated with the aid of Synthetic Aperture Radar (SAR) images that can penetrate cloud cover. However, the large domain gap between optical and SAR images as well as the severe speckle noise of SAR images may cause significant interference in SAR-based cloud removal, resulting in performance degeneration. In this paper, we propose a novel global-local fusion based cloud removal (GLF-CR) algorithm to leverage the complementary information embedded in SAR images. Exploiting the power of SAR information to promote cloud removal entails two aspects. The first, global fusion, guides the relationship among all local optical windows to maintain the structure of the recovered region consistent with the remaining cloudfree regions. The second, local fusion, transfers complementary information embedded in the SAR image that corresponds to cloudy areas to generate reliable texture details of the missing regions, and uses dynamic filtering to alleviate the performance degradation caused by speckle noise. Extensive evaluation demonstrates that the proposed algorithm can yield high quality cloud-free images and performs favorably against state-of-the-art cloud removal algorithms.

show abstract

SEN12MS-CR-TS: A Remote-Sensing Data Set for Multimodal Multitemporal Cloud Removal

Cited by 78 publications

References 46 publications

Supervised Change Detection Using Prechange Optical-SAR and Postchange SAR Data

Supervised Change Detection Using Prechange Optical-SAR and Postchange SAR Data

Explaining the Effects of Clouds on Remote Sensing Scene Classification

Exploring the Potential of SAR Data for Cloud Removal in Optical Satellite Imagery

Contact Info

Product

Resources

About