FMWDCT: Foreground Mixup Into Weighted Dual-Network Cross Training for Semisupervised Remote Sensing Road Extraction

Abstract

With the development of deep learning, the application of automatic road extraction has achieved great success. However, the main challenge is how to make full use of a large number of unlabeled images to improve segmentation models and how alleviate sample imbalance in road extraction tasks. In this paper, we propose a novel semi-supervised remote sensing road extraction approach is refined as Foreground Mixup into Weighted Dual-network Cross Training (FMWDCT), which combines labeled images with unlabeled images to extract road from remote sensing images. FMWDCT is composed of Dualnetwork Cross Training (DCT) and Foreground Pasting (FP). DCT is a new semi-supervised training method and FP is an effective data perturbation method for road extraction. We firstly paste the foreground pixels obtained from labeled images into unlabeled images to produce mixed input images. The mixed pseudo labels are then generated by a combination of highconfidence predictions from the augmented network and labeled masks. Finally, the mixed pseudo labels are used to guide another adversarial basic network for cross training, and this basic network is used to smoothly update the augmented network that corresponds to it. The proposed FMWDCT effectively solve the overfitting problem and imbalance problem of positive and negative sample in the case of a few labeled training samples. We demonstrate the effectiveness of our method on three road extraction datasets, and achieve better performance with few labeled data. Extensive experiments show that the proposed semi-supervised method can learn latent information from the unlabeled data to improve performance.

A Survey of Deep Learning Road Extraction Algorithms Using High-Resolution Remote Sensing Images

“…The concept of consistency regularization and pseudo-labeling was introduced into semi-supervised road extraction tasks by You et al [ 149 ], who proposed a novel semi-supervised remote sensing road extraction method called “FMWDCT”. This method comprises two key components: dual-network cross training (DCT) and foreground pasting (FP).…”

Semisupervised Edge-Aware Road Extraction via Cross Teaching Between CNN and Transformer

“…We demonstrate the effectiveness of our method by comparing it with four state-of-the-art methods on three publicly accessible datasets, all of which include partially labeled data. Regarding the semi-supervised experiment in FMWDCT [36], the DeepGlobe Dataset is categorized into four different III.…”

On the Effects of Different Types of Label Noise in Multi-Label Remote Sensing Image Classification

“…Further, Zhang et al [36] propose Object First Mixup, an alteration to Mixup that adjusts the weights of segmented objects and their background in semisupervised object detection. Finally, in semantic segmentation, You et al [39] incorporate the copy-paste mechanism (which is the pixel annotation-based equivalent of CutMix) into a semi-supervised framework in which the ground reference pixels of annotated roads are pasted onto unlabeled images. Concurrently, the framework RanPaste introduced by Wang et al [10] exploits color jitter and random Gaussian noise for consistency regularization while making use of a CutMix-like pasting mechanism that pastes a bounding box (in contrast to pixels of a segment) of a labeled image onto an unlabeled image.…”

“…Similarly to CV, data augmentation techniques significantly improve the performance and generalization capabilities for RS tasks. Recently, researchers have explored various approaches to tailor well-established CV data augmentation techniques for RS tasks, such as SLC [8], [30], [31], [32], [33], object detection [9], [34], [35], [36], [37], [38], or semantic segmentation tasks [10], [39]. Joined in their purpose to alleviate overfitting and to serve as a form of regularization in small data regimes, the approaches for data augmentation can be broadly divided into three categories: (i) image modification (Section II-A); (ii) image generation (Section II-B); or (iii) image pairing (Section II-C) that also includes CutMix-based approaches.…”