Machine-learned Regularization and Polygonization of Building Segmentation Masks

Zorzi, Stefano; Bittner, Ksenia; Fraundorfer, Friedrich

doi:10.1109/icpr48806.2021.9412866

Cited by 43 publications

(37 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These rankings persist for each of the two benchmark datasets as well, suggesting that the results are somewhat robust to variations in the underlying data. These results are comparable to current state-ofthe-art results on Inria [8,26,51]. The results tentatively suggest that including some transformer modules (e.g., the TransUnet) is beneficial, however including too much can be detrimental (e.g., SwinUnet).…”

Section: Model Performance Comparisonssupporting

confidence: 83%

Transformers For Recognition In Overhead Imagery: A Reality Check

Luzi¹,

Gupta²,

Collins³

et al. 2022

Preprint

View full text Add to dashboard Cite

There is evidence that transformers offer state-of-theart recognition performance on tasks involving overhead imagery (e.g., satellite imagery). However, it is difficult to make unbiased empirical comparisons between competing deep learning models, making it unclear whether, and to what extent, transformer-based models are beneficial. In this paper we systematically compare the impact of adding transformer structures into state-of-the-art segmentation models for overhead imagery. Each model is given a similar budget of free parameters, and their hyperparameters are optimized using Bayesian Optimization with a fixed quantity of data and computation time. We conduct our experiments with a large and diverse dataset comprising two large public benchmarks: Inria and DeepGlobe. We perform additional ablation studies to explore the impact of specific transformer-based modeling choices. Our results suggest that transformers provide consistent, but modest, performance improvements. We only observe this advantage however in hybrid models that combine convolutional and transformer-based structures, while fully transformerbased models achieve relatively poor performance.

show abstract

Section: Model Performance Comparisonssupporting

confidence: 83%

Transformers For Recognition In Overhead Imagery: A Reality Check

Luzi¹,

Gupta²,

Collins³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Zorzi et al [41] use FCN trained with a combination of adversarial and regularized losses to perform building boundary refinement and regularization. Zorzi et al [42] extend this algorithm by using a GANbased model to extract regularized building boundaries after obtaining the building mask using FCN.…”

Section: Change Detection Methods Based On Deep Learningmentioning

confidence: 99%

Feature Decomposition-Optimization-Reorganization Network for Building Change Detection in Remote Sensing Images

Zhou

Zhu

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

Building change detection plays an imperative role in urban construction and development. Although the deep neural network has achieved tremendous success in remote sensing image building change detection, it is still fraught with the problem of generating broken detection boundaries and separation of dense buildings, which tends to produce saw-tooth boundaries. In this work, we propose a feature decomposition-optimization-reorganization network for building change detection. The main contribution of the proposed network is that it performs change detection by respectively modeling the main body and edge features of buildings, which is based on the characteristics that the similarity between the main body pixels is strong but weak between the edge pixels. Firstly, we employ a siamese ResNet structure to extract dual-temporal multi-scale difference features on the original remote sensing images. Subsequently, a flow field is built to separate the main body and edge features. Thereafter, a feature optimization module is designed to refine the main body and edge features using the main body and edge ground truth. Finally, we reorganize the optimized main body and edge features to obtain the output results. These constitute a complete end-to-end building change detection framework. The publicly available building dataset LEVIR-CD is employed to evaluate the change detection performance of our network. The experimental results show that the proposed method can accurately identify the boundaries of changed buildings, and obtain better results compared with the current state-of-the-art methods based on the U-Net structure or by combining spatial-temporal attention mechanisms.

show abstract

“…Their performance heavily depends on the quality of the segmentation map, which deteriorates seriously when the segmentation map is not perfect. (Zorzi, Bittner, and Fraundorfer 2020) designed an approach to regularize the building segmentation maps via a generative adversarial network, which requires a multi-stage training procedure for optimizing different network components.…”

Section: Polygonal Instance Segmentationmentioning

confidence: 99%

“…These methods not only require a complex processing procedure, but also a perfect segmentation map to ensure the quality of the polygonization results. To solve the above limitations, a generative adversarial network based method was proposed in (Zorzi, Bittner, and Fraundorfer 2020) for regularizing the building segmentation maps. Although producing visually pleasing building polygons, the method consists of three separate networks and requires heavy training procedures regarding the hybrid losses of different network components.…”

Section: Introductionmentioning

confidence: 99%

Joint Semantic-geometric Learning for Polygonal Building Segmentation

Zhao

Zhong

et al. 2021

AAAI

View full text Add to dashboard Cite

Building extraction from aerial or satellite images has been an important research issue in remote sensing and computer vision domains for decades. Compared with pixel-wise semantic segmentation models that output raster building segmentation map, polygonal building segmentation approaches produce more realistic building polygons that are in the desirable vector format for practical applications. Despite the substantial efforts over recent years, state-of-the-art polygonal building segmentation methods still suffer from several limitations, e.g., (1) relying on a perfect segmentation map to guarantee the vectorization quality; (2) requiring a complex post-processing procedure; (3) generating inaccurate vertices with a fixed quantity, a wrong sequential order, self-intersections, etc. To tackle the above issues, in this paper, we propose a polygonal building segmentation approach and make the following contributions: (1) We design a multi-task segmentation network for joint semantic and geometric learning via three tasks, i.e., pixel-wise building segmentation, multi-class corner prediction, and edge orientation prediction. (2) We propose a simple but effective vertex generation module for transforming the segmentation contour into high-quality polygon vertices. (3) We further propose a polygon refinement network that automatically moves the polygon vertices into more accurate locations. Results on two popular building segmentation datasets demonstrate that our approach achieves significant improvements for both building instance segmentation (with 2% F1-score gain) and polygon vertex prediction (with 6% F1-score gain) compared with current state-of-the-art methods.

show abstract

Machine-learned Regularization and Polygonization of Building Segmentation Masks

Cited by 43 publications

References 32 publications

Transformers For Recognition In Overhead Imagery: A Reality Check

Transformers For Recognition In Overhead Imagery: A Reality Check

Feature Decomposition-Optimization-Reorganization Network for Building Change Detection in Remote Sensing Images

Joint Semantic-geometric Learning for Polygonal Building Segmentation

Contact Info

Product

Resources

About