Interactive Correction of a Distorted Street-View Panorama for Efficient 3-D Façade Modeling

Zhu, Qing; Zhang, Mier; Han, Hu; Wang, Feng

doi:10.1109/lgrs.2019.2962696

Cited by 14 publications

(8 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The detection and segmentation approaches above generally cannot enforce the geometry regularities of the façade objects, such as alignment and equal-size constraints. A straightforward approach for regularization explicitly offsets adjacent points or boundaries in a least-squares optimization (Arikan et al, 2013;Zhu et al, 2020b;Xie et al, 2018). Because least-squares optimization cannot model the logic operators, such as sameColumn and sameWidth (Dehbi et al, 2017), clustering (Fan et al, 2021), learned Markov Logic Networks (Dehbi et al, 2017), or binary integer programming (Hu et al, 2020;Hensel et al, 2019;Monszpart et al, 2015) are exploited to snap the feature points on the façade.…”

Section: Related Workmentioning

confidence: 99%

Semi-Supervised Adversarial Recognition of Refined Window Structures for Inverse Procedural Façade Modeling

Huang¹,

Liang²,

Ding³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning methods are notoriously data-hungry, which requires a large number of labeled samples. Unfortunately, the large amount of interactive sample labeling efforts has dramatically hindered the application of deep learning methods, especially for 3D modeling tasks, which require heterogeneous samples. To alleviate the work of data annotation for learned 3D modeling of façades, this paper proposed a semi-supervised adversarial recognition strategy embedded in inverse procedural modeling. Beginning with textured LOD-2 (Level-of-Details) models, we use the classical convolutional neural networks to recognize the types and estimate the parameters of windows from image patches. The window types and parameters are then assembled into procedural grammar. A simple procedural engine is built inside an existing 3D modeling software, producing fine-grained window geometries. To obtain a useful model from a few labeled samples, we leverage the generative adversarial network to train the feature extractor in a semi-supervised manner. The adversarial training strategy can also exploit unlabeled data to make the training phase more stable. Experiments using publicly available façade image datasets reveal that the proposed training strategy can obtain about 10% improvement in classification accuracy and 50% improvement in parameter estimation under the same network structure. In addition, performance gains are more pronounced when testing against unseen data featuring different façade styles.

show abstract

Section: Related Workmentioning

confidence: 99%

Semi-Supervised Adversarial Recognition of Refined Window Structures for Inverse Procedural Façade Modeling

Huang¹,

Liang²,

Ding³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, lines or corners-the most salient features of building façades-are typically used (Nyaruhuma et al, 2010). Texture information (Frommholz et al, 2015;Yang et al, 2015;Zhu et al, 2020c;Hu et al, 2016a) can also be used to parse façades, and roofs and façades can be jointly considered to improve robustness (Xiao and Gerke, 2015).…”

Section: Related Workmentioning

confidence: 99%

Depth-Enhanced Feature Pyramid Network for Occlusion-Aware Verification of Buildings from Oblique Images

Zhu,

Huang,

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Detecting the changes of buildings in urban environments is essential. Existing methods that use only nadir images suffer from severe problems of ambiguous features and occlusions between buildings and other regions. Furthermore, buildings in urban environments vary significantly in scale, which leads to performance issues when using single-scale features. To solve these issues, this paper proposes a fused feature pyramid network, which utilizes both color and depth data for the 3D verification of existing buildings 2D footprints from oblique images. First, the color data of oblique images are enriched with the depth information rendered from 3D mesh models. Second, multiscale features are fused in the feature pyramid network to convolve both the color and depth data. Finally, multi-view information from both the nadir and oblique images is used in a robust voting procedure to label changes in existing buildings. Experimental evaluations using both the ISPRS benchmark datasets and Shenzhen datasets reveal that the proposed method outperforms the ResNet and EfficientNet networks by 5% and 2%, respectively, in terms of recall rate and precision. We demonstrate that the proposed method can successfully detect all changed buildings; therefore, only those marked as changed need to be manually checked during the pipeline updating procedure; this significantly reduces the manual quality control requirements. Moreover, ablation studies indicate that using depth data, feature pyramid modules, and multi-view voting strategies can lead to clear and progressive improvements.

show abstract

“…The use of thermal imager equipment, requires a large of labor and time costs in large-scale modeling. By improving the distortion correction method of panoramic images, Zhu et al (2020) excavate reliable information of building facades from panoramic photos. However, this method does not fundamentally solve noisy problem in images.…”

Section: Introductionmentioning

confidence: 99%

Layout graph model for semantic façade reconstruction using laser point clouds

Fan

Wang

Gong

2021

Geo-spatial Information Science

View full text Add to dashboard Cite

Building façades can feature different patterns depending on the architectural style, functionality, and size of the buildings; therefore, reconstructing these façades can be complicated. In particular, when semantic façades are reconstructed from point cloud data, uneven point density and noise make it difficult to accurately determine the façade structure. When investigating façade layouts, Gestalt principles can be applied to cluster visually similar floors and façade elements, allowing for a more intuitive interpretation of façade structures. We propose a novel model for describing façade structures, namely the layout graph model, which involves a compound graph with two structure levels. In the proposed model, similar façade elements such as windows are first grouped into clusters. A down-layout graph is then formed using this cluster as a node and by combining intra-and inter-cluster spacings as the edges. Second, a top-layout graph is formed by clustering similar floors. By extracting relevant parameters from this model, we transform semantic façade reconstruction to an optimization strategy using simulated annealing coupled with Gibbs sampling. Multiple façade point cloud data with different features were selected from three datasets to verify the effectiveness of this method. The experimental results show that the proposed method achieves an average accuracy of 86.35%. Owing to its flexibility, the proposed layout graph model can deal with different types of façades and qualities of point cloud data, enabling a more robust and accurate reconstruction of façade models.

show abstract

Interactive Correction of a Distorted Street-View Panorama for Efficient 3-D Façade Modeling

Cited by 14 publications

References 16 publications

Semi-Supervised Adversarial Recognition of Refined Window Structures for Inverse Procedural Façade Modeling

Semi-Supervised Adversarial Recognition of Refined Window Structures for Inverse Procedural Façade Modeling

Depth-Enhanced Feature Pyramid Network for Occlusion-Aware Verification of Buildings from Oblique Images

Layout graph model for semantic façade reconstruction using laser point clouds

Contact Info

Product

Resources

About