Joint Semantic-geometric Learning for Polygonal Building Segmentation

Abstract

Building extraction from aerial or satellite images has been an important research issue in remote sensing and computer vision domains for decades. Compared with pixel-wise semantic segmentation models that output raster building segmentation map, polygonal building segmentation approaches produce more realistic building polygons that are in the desirable vector format for practical applications. Despite the substantial efforts over recent years, state-of-the-art polygonal building segmentation methods still suffer from several limitations, e.g., (1) relying on a perfect segmentation map to guarantee the vectorization quality; (2) requiring a complex post-processing procedure; (3) generating inaccurate vertices with a fixed quantity, a wrong sequential order, self-intersections, etc. To tackle the above issues, in this paper, we propose a polygonal building segmentation approach and make the following contributions: (1) We design a multi-task segmentation network for joint semantic and geometric learning via three tasks, i.e., pixel-wise building segmentation, multi-class corner prediction, and edge orientation prediction. (2) We propose a simple but effective vertex generation module for transforming the segmentation contour into high-quality polygon vertices. (3) We further propose a polygon refinement network that automatically moves the polygon vertices into more accurate locations. Results on two popular building segmentation datasets demonstrate that our approach achieves significant improvements for both building instance segmentation (with 2% F1-score gain) and polygon vertex prediction (with 6% F1-score gain) compared with current state-of-the-art methods.

Multi-Layer Fault-Tolerant Protection Strategies for Hybrid Distribution Transformers Integrated Photovoltaic Systems

“…These methods output vertex step by step, leading to sensitivity to complex buildings. To avoid iteratively predicting serialized vertices, some work [25], [29], [43]- [47] first sample serialized vertices in a single direction from predicted segmentation masks and update vertex positions several times using different refinement modules. PolyBuilding [48] introduces vertices regression and classification heads to directly predict serialized vertices of a building, but it still needs thresholds to control the vertex number.…”

Transferring Transformer-Based Models for Cross-Area Building Extraction From Remote Sensing Images

“…The data of different cities is released in separate folders instead of combined, making it possible for cross-area tests. In literature, 337 usually only a part of the dataset, e.g., only available data form 338 one or two cities, is used, in which case, the training and test 339 split is usually in a random format [58], [27].…”

Vector Mapping Method for Buildings in Remote Sensing Images Based on Joint Semantic-Geometric Learning

“…High-resolution remote sensing images have become a primary data source for the vector mapping of building polygons as remote sensing and earth observation technology have advanced [3]. Automatic building outline extraction from high-resolution remote sensing images is an important method for improving vector mapping efficiency [4], [5] and has been both the focus and challenge of remote sensing applications and cartographic research [6]- [8].…”