MFTSC: A Semantically Constrained Method for Urban Building Height Estimation Using Multiple Source Images

Abstract

The use of remote sensing imagery has significantly enhanced the efficiency of building extraction; however, the precise estimation of building height remains a formidable challenge. In light of ongoing advancements in computer vision, numerous techniques leveraging convolutional neural networks and Transformers have been applied to remote sensing imagery, yielding promising outcomes. Nevertheless, most existing approaches directly estimate height without considering the intrinsic relationship between semantic building segmentation and building height estimation. In this study, we present a unified architectural framework that integrates the tasks of building semantic segmentation and building height estimation. We introduce a Transformer model that systematically merges multi-level features with semantic constraints and leverages shallow spatial detail feature cues in the encoder. Our approach excels in both height estimation and semantic segmentation tasks. Specifically, the coefficient of determination (R2) in the height estimation task attains a remarkable 0.9671, with a root mean square error (RMSE) of 1.1733 m. The mean intersection over union (mIoU) for building semantic segmentation reaches 0.7855. These findings underscore the efficacy of multi-task learning by integrating semantic segmentation with height estimation, thereby enhancing the precision of height estimation.

FusionHeightNet: A Multi-Level Cross-Fusion Method from Multi-Source Remote Sensing Images for Urban Building Height Estimation

“…Compared with other sensors, such as LiDAR [8], the acquisition cost of remote sensing images is obviously lower. Therefore, estimating building heights from remote sensing images has become a credible idea [9]. However, since a single image may correspond to countless height structures, estimating the height of buildings from a single remote sensing image is a challenging problem.…”

“…In urban areas with both optical images and SAR images, improving the precision of building height estimation by fusing optical images and SAR images has begun to attract attention. Although there have been some studies [9,12] aiming to fuse SAR images and optical images to estimate building heights, these methods ignore the exploration of fusion methods adopting SAR images and optical images, which simply combine the features of the two modalities. This simple combination of heterogeneous features often leads to a decline in results and fails to achieve the purpose of complementing the feature information of the two modalities.…”

From Segmentation to Classification: A Deep Learning Scheme for Sintered Surface Images Processing

“…In recent years, deep learning has demonstrated powerful visual feature learning abilities [10][11][12][13][14]. A large number of deep learning methods are applied in object detection [15], semantic segmentation [16], image classification [17], recommender systems [18] and medical image analysis [19]. However, the application of deep learning methods for downstream tasks in sintering industrial scenarios have some challenges.…”

MSSFF: Advancing Hyperspectral Classification through Higher-Accuracy Multistage Spectral–Spatial Feature Fusion

“…In recent years, with the advancements in deep learning, this technology has been increasingly adopted in various domains [9][10][11], including hyperspectral remote sensing, and has achieved remarkable success [6]. Deep learning models have the capability to extract meaningful knowledge from vast amounts of redundant data [12].…”