Dense Dilated Convolutions’ Merging Network for Land Cover Classification

Abstract

Land cover classification of remote sensing images is a challenging task due to limited amounts of annotated data, highly imbalanced classes, frequent incorrect pixel-level annotations, and an inherent complexity in the semantic segmentation task. In this work, we propose a novel architecture called the Dense Dilated Convolutions Merging Network (DDCM-Net) to address this task. The proposed DDCM-Net consists of dense dilated image convolutions merged with varying dilation rates. This effectively utilizes rich combinations of dilated convolutions that enlarge the network's receptive fields with less parameters and features compared to the state-of-the-art approaches in the remote sensing domain. Importantly, DDCM-Net obtains fused local and global context information, in effect incorporating surrounding discriminative capability for multi-scale and complex shaped objects with similar color and textures in very high resolution aerial imagery. We demonstrate the effectiveness, robustness and flexibility of the proposed DDCM-Net on the publicly available ISPRS Potsdam and Vaihingen data, as well as the DeepGlobe land cover dataset. Our single model, trained on 3-band Potsdam and Vaihingen data, achieves better accuracy in terms of both mean intersection over union (mIoU) and F1-score compared to other published models trained with more than 3band data. We further validate our model on the DeepGlobe dataset, achieving state-of-the-art result 56.2% mIoU with much less parameters and at a lower computational cost compared to related recent work.

CSE-HRNet: A Context and Semantic Enhanced High-Resolution Network for Semantic Segmentation of Aerial Imagery

“…Tree-UNet [22] adaptively constructs the Tree-shape convolutional blocks though the Tree-Cutting algorithm to fuse the multi-scale features and learn the best weights. DDCM-Net [46] consists of combinations of the dilated convolutions merged with varying dilation rates to enlarge the network's receptive fields. Moreover, ENRU-Net [23] and DRR-Net [47] adopt the encoder-decoder structures to automatically segment diversified buildings and roads from aerial imagery data, respectively.…”

“…The dilated convolution is an effective method to increase the receptive field and learn global context features without any reduction in the feature map resolution [30]. As mentioned, it is widely used in various semantic segmentation models to aggregate multi-scale contextual information, such as DeepLab variants [30]- [32], PSPNet [33], DenseA-SPP [34], DRN [35], WASP [36], KSAC [37], ScasNet [40], and DDCM-Net [46].…”

“…These two datasets contain the high-resolution true ortho photo (TOP), the digital surface model (DSM), and the normalized DSM (nDSM) data, with the corresponding ground truth labels. While both DSM and nDSM data are included in the two datasets, we only focused on the raw TOP images in this work, following [40], [43], [46].…”

“…Image tiles consist of four spectral bands: red (R), green (G), blue (B), and near infrared (IR). But we only focused on the three-channel RGB tiles in this work, as in [46]. The Vaihingen dataset consists of 33 TOP image tiles of varying sizes (on average approximately 2100 × 2100 pixels), which have a spatial resolution of 9 cm.…”

“…When making inferences on the test set, we adopted a sliding window to traverse the whole images to yield seamless prediction results. Following [19], [25], [46], in order to obtain well-smoothed inference results, consecutive overlapping patches were retained in the test data (50% overlapping for Potsdam, and 75% for Vaihingen) in our experiments, and then multiple predictions were averaged to smooth the prediction of boundaries and eliminate the possible discontinuities.…”