Unsupervised Deep Transfer Learning-Based Change Detection for HR Multispectral Images

Saha, Sudipan; Solano‐Correa, Yady Tatiana; Bovolo, Francesca; Bruzzone, Lorenzo

doi:10.1109/lgrs.2020.2990284

Cited by 50 publications

(44 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thresholding is done using Otsu's method [43], as it is popular in unsupervised CD methods [19], [48]. However, any other suitable method can be used, as shown in Table V.…”

Section: ) Las Vegasmentioning

confidence: 99%

Self-Supervised Multisensor Change Detection

Saha

Ebel

Zhu

2022

IEEE Trans. Geosci. Remote Sensing

Self Cite

View full text Add to dashboard Cite

Most change detection (CD) methods assume that prechange and postchange images are acquired by the same sensor. However, in many real-life scenarios, e.g., natural disasters, it is more practical to use the latest available images before and after the occurrence of incidence, which may be acquired using different sensors. In particular, we are interested in the combination of the images acquired by optical and synthetic aperture radar (SAR) sensors. SAR images appear vastly different from the optical images even when capturing the same scene. Adding to this, CD methods are often constrained to use only target image-pair, no labeled data, and no additional unlabeled data. Such constraints limit the scope of traditional supervised machine learning and unsupervised generative approaches for multisensor CD. The recent rapid development of self-supervised learning methods has shown that some of them can even work with only few images. Motivated by this, in this work, we propose a method for multisensor CD using only the unlabeled target bitemporal images that are used for training a network in a self-supervised fashion by using deep clustering and contrastive learning. The proposed method is evaluated on four multimodal bitemporal scenes showing change, and the benefits of our self-supervised approach are demonstrated. Code is available at https://gitlab.lrz.de/ai4eo/cd/-/tree/main/sarOpticalMultisensorTgrs2021.

show abstract

“…Thresholding is done using Otsu's method [43], as it is popular in unsupervised CD methods [19], [48]. However, any other suitable method can be used, as shown in Table V.…”

Section: ) Las Vegasmentioning

confidence: 99%

Self-Supervised Multisensor Change Detection

Saha

Ebel

Zhu

2022

IEEE Trans. Geosci. Remote Sensing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently, a cycle-consistent GAN has been exploited to transcode synthetic-aperture-radar (SAR) images into optical images for building change detection using optical-like features [17,18]. GAN architectures have also proven their advantages using discriminative features of the discriminator in hyperspectral image classification [19,20]. Unsupervised image-to-image translation has been employed for domain adaptation in aerial image segmentation [21].…”

Section: Related Workmentioning

confidence: 99%

Raindrop-Aware GAN: Unsupervised Learning for Raindrop-Contaminated Coastal Video Enhancement

et al. 2020

View full text Add to dashboard Cite

We propose an unsupervised network with adversarial learning, the Raindrop-aware GAN, which enhances the quality of coastal video images contaminated by raindrops. Raindrop removal from coastal videos faces two main difficulties: converting the degraded image into a clean one by visually removing the raindrops, and restoring the background coastal wave information in the raindrop regions. The components of the proposed network—a generator and a discriminator for adversarial learning—are trained on unpaired images degraded by raindrops and clean images free from raindrops. By creating raindrop masks and background-restored images, the generator restores the background information in the raindrop regions alone, preserving the input as much as possible. The proposed network was trained and tested on an open-access dataset and directly collected dataset from the coastal area. It was then evaluated by three metrics: the peak signal-to-noise ratio, structural similarity, and a naturalness-quality evaluator. The indices of metrics are 8.2% (+2.012), 0.2% (+0.002), and 1.6% (−0.196) better than the state-of-the-art method, respectively. In the visual assessment of the enhanced video image quality, our method better restored the image patterns of steep wave crests and breaking than the other methods. In both quantitative and qualitative experiments, the proposed method more effectively removed the raindrops in coastal video and recovered the damaged background wave information than state-of-the-art methods.

show abstract

“…In addition, with the advances of machine learning during the last years, more and more techniques based on neural networks are emerging [19], [20], [21], [22], [23], [24], [25] aiming to create robust systems that can successfully tackle the change detection problem. Among the machine learning approaches, deep learning architectures are the ones that have captured most of the attention owing to state of the art on numerous computer vision applications [26], [27], [28], [29] including remote sensing [30], [31], [32], [33], [34], [35], [36], [37]. Even though successful results have been achieved on the remote sensing domain, the further development of deep neural networks is still hindered owing to insufficient datasets lacking multi-modal diverse information.…”

Section: Introductionmentioning

confidence: 99%

A Deep Multitask Learning Framework Coupling Semantic Segmentation and Fully Convolutional LSTM Networks for Urban Change Detection

Papadomanolaki

Vakalopoulou

Καράντζαλος

2021

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

In this paper, we present a deep multi-task learning framework able to couple semantic segmentation and change detection using fully convolutional long short-term memory (LSTM) networks. In particular, we present a UNet-like architecture (L-UNet) which models the temporal relationship of spatial feature representations using integrated fully convolutional LSTM blocks on top of every encoding level. In this way, the network is able to capture the temporal relationship of spatial feature vectors in all encoding levels without the need to downsample or flatten them, forming an end-to-end trainable framework. Moreover, we further enrich the L-UNet architecture with an additional decoding branch that performs semantic segmentation on the available semantic categories that are presented in the different input dates, forming a multi-task framework. Different loss quantities are also defined and combined together in a circular way to boost the overall performance. The developed methodology has been evaluated on three different datasets, i.e, the challenging bi-temporal high-resolution ONERA Satellite Change Detection (OSCD) Sentinel-2 dataset, the very high-resolution multitemporal dataset of the East Prefecture of Attica, Greece, and lastly, the multitemporal very high-resolution SpaceNet7 dataset. Promising quantitative and qualitative results demonstrated that the synergy among the tasks can boost up the achieved performances. In particular, the proposed multi-task framework contributed to a significant decrease of false positive detections, with F1 rate outperforming other state of the art methods by at least 2.1% and 4.9% in the Attica VHR and SpaceNet7 dataset case respectively. Our models and code can be found at: https://github.com/mpapadomanolaki/multi-task-L-UNet

show abstract

Unsupervised Deep Transfer Learning-Based Change Detection for HR Multispectral Images

Cited by 50 publications

References 10 publications

Self-Supervised Multisensor Change Detection

Self-Supervised Multisensor Change Detection

Raindrop-Aware GAN: Unsupervised Learning for Raindrop-Contaminated Coastal Video Enhancement

A Deep Multitask Learning Framework Coupling Semantic Segmentation and Fully Convolutional LSTM Networks for Urban Change Detection

Contact Info

Product

Resources

About