Building Extraction from High Spatial Resolution Remote Sensing Images via Multiscale-Aware and Segmentation-Prior Conditional Random Fields

Zhu, Qiqi; Li, Zhen; Zhang, Yanan; Guan, Qingfeng

doi:10.3390/rs12233983

Cited by 52 publications

(21 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, there are many papers to improve the building extraction performance by focusing on network architecture designing. Structures such as deep and shallow feature fusion [8,38,[46][47][48][49][50][51][52][53][54][55], multiple receptive field [5,12,48,51,[54][55][56][57], residual connection [1,11,47,51,52,[57][58][59] have been widely used in building extraction. MAP-Net [46] alleviates the scale problem by capturing spatial localization-preserved multi-scale features through a multi-parallel path design.…”

Section: Cnn-based Methods For Building Extractionmentioning

confidence: 99%

Building Extraction from Remote Sensing Images with Sparse Token Transformers

2021

View full text Add to dashboard Cite

Deep learning methods have achieved considerable progress in remote sensing image building extraction. Most building extraction methods are based on Convolutional Neural Networks (CNN). Recently, vision transformers have provided a better perspective for modeling long-range context in images, but usually suffer from high computational complexity and memory usage. In this paper, we explored the potential of using transformers for efficient building extraction. We design an efficient dual-pathway transformer structure that learns the long-term dependency of tokens in both their spatial and channel dimensions and achieves state-of-the-art accuracy on benchmark building extraction datasets. Since single buildings in remote sensing images usually only occupy a very small part of the image pixels, we represent buildings as a set of “sparse” feature vectors in their feature space by introducing a new module called “sparse token sampler”. With such a design, the computational complexity in transformers can be greatly reduced over an order of magnitude. We refer to our method as Sparse Token Transformers (STT). Experiments conducted on the Wuhan University Aerial Building Dataset (WHU) and the Inria Aerial Image Labeling Dataset (INRIA) suggest the effectiveness and efficiency of our method. Compared with some widely used segmentation methods and some state-of-the-art building extraction methods, STT has achieved the best performance with low time cost.

show abstract

Section: Cnn-based Methods For Building Extractionmentioning

confidence: 99%

Building Extraction from Remote Sensing Images with Sparse Token Transformers

2021

View full text Add to dashboard Cite

show abstract

“…In order to leverage large-scale contextual information and extract critical cues for identifying building pixels in the presence of complex background and when there is occlusion, researchers have proposed methods to capture local and longrange spatial dependencies among the ground entities in the aerial scene [55], [56]. Several researchers are also using tranformers [60], attention modules [12], [61]- [63] and multiscale information [8], [43], [45], [46], [64]- [66] for this purpose. Recently, multi-view satellite images [67], [68] are also being used to perform semantic segmentation of points on ground.…”

Section: Related Workmentioning

confidence: 99%

“…The metrics do not enforce the requirement of contiguity of the pixels that belong to the same building [1]- [6]. This has led some researchers to formulate post-processing steps like the Conditional Random Fields (CRFs) [7], [8] during inference for invoking spatial contiguity in the output label maps.…”

Section: Introductionmentioning

confidence: 99%

Uncertainty, Edge, and Reverse-Attention Guided Generative Adversarial Network for Automatic Building Detection in Remotely Sensed Images

Chattopadhyay¹,

Kak²

2021

Preprint

View full text Add to dashboard Cite

Despite recent advances in deep-learning based semantic segmentation, automatic building detection from remotely sensed imagery is still a challenging problem owing to large variability in the appearance of buildings across the globe. The errors occur mostly around the boundaries of the building footprints, in shadow areas, and when detecting buildings whose exterior surfaces have reflectivity properties that are very similar to those of the surrounding regions. To overcome these problems, we propose a generative adversarial network based segmentation framework with uncertainty attention unit and refinement module embedded in the generator. The refinement module, composed of edge and reverse attention units, is designed to refine the predicted building map. The edge attention enhances the boundary features to estimate building boundaries with greater precision, and the reverse attention allows the network to explore the features missing in the previously estimated regions. The uncertainty attention unit assists the network in resolving uncertainties in classification. As a measure of the power of our approach, as of December 4, 2021, it ranks at the second place on DeepGlobe's public leaderboard despite the fact that main focus of our approach -refinement of the building edges -does not align exactly with the metrics used for leaderboard rankings. Our overall F1-score on DeepGlobe's challenging dataset is 0.745. We also report improvements on the previous-best results for the challenging INRIA Validation Dataset for which our network achieves an overall IoU of 81.28% and an overall accuracy of 97.03%. Along the same lines, for the official INRIA Test Dataset, our network scores 77.86% and 96.41% in overall IoU and accuracy. We have also improved upon the previous best results on two other datasets: For the WHU Building Dataset, our network achieves 92.27% IoU, 96.73% precision, 95.24% recall and 95.98% F1-score. And, finally, for the Massachusetts Buildings Dataset, our network achieves 96.19% relaxed IoU score and 98.03% relaxed F1-score over the previous best scores of 91.55% and 96.78% respectively, and in terms of non-relaxed F1 and IoU scores, our network outperforms the previous best scores by 2.77% and 3.89% respectively.

show abstract

“…The second is data improvement, which consists of establishing a high-quality and high-precision sample set, increasing sample data in the study field, and realizing sample data improvement by fusing multisource data like DSM [21,22]. The third is classifier synthesis, which introduces the conditional random field and attention mechanism to improve classification accuracy [23,24]. While the encoding and decoding structure in FCN can realize an end-to-end network structure, the original image information lost during the encoding phase is difficult to recover during the decoding phase, resulting in the fuzzy edges of building extraction results, a loss of building details, and building details a reduction in extraction accuracy.…”

Section: Introductionmentioning

confidence: 99%

Building Multi-Feature Fusion Refined Network for Building Extraction from High-Resolution Remote Sensing Images

et al. 2021

View full text Add to dashboard Cite

Deep learning approaches have been widely used in building automatic extraction tasks and have made great progress in recent years. However, the missing detection and wrong detection causing by spectrum confusion is still a great challenge. The existing fully convolutional networks (FCNs) cannot effectively distinguish whether the feature differences are from one building or the building and its adjacent non-building objects. In order to overcome the limitations, a building multi-feature fusion refined network (BMFR-Net) was presented in this paper to extract buildings accurately and completely. BMFR-Net is based on an encoding and decoding structure, mainly consisting of two parts: the continuous atrous convolution pyramid (CACP) module and the multiscale output fusion constraint (MOFC) structure. The CACP module is positioned at the end of the contracting path and it effectively minimizes the loss of effective information in multiscale feature extraction and fusion by using parallel continuous small-scale atrous convolution. To improve the ability to aggregate semantic information from the context, the MOFC structure performs predictive output at each stage of the expanding path and integrates the results into the network. Furthermore, the multilevel joint weighted loss function effectively updates parameters well away from the output layer, enhancing the learning capacity of the network for low-level abstract features. The experimental results demonstrate that the proposed BMFR-Net outperforms the other five state-of-the-art approaches in both visual interpretation and quantitative evaluation.

show abstract

Building Extraction from High Spatial Resolution Remote Sensing Images via Multiscale-Aware and Segmentation-Prior Conditional Random Fields

Cited by 52 publications

References 43 publications

Building Extraction from Remote Sensing Images with Sparse Token Transformers

Building Extraction from Remote Sensing Images with Sparse Token Transformers

Uncertainty, Edge, and Reverse-Attention Guided Generative Adversarial Network for Automatic Building Detection in Remotely Sensed Images

Building Multi-Feature Fusion Refined Network for Building Extraction from High-Resolution Remote Sensing Images

Contact Info

Product

Resources

About