High Quality Facade Segmentation Based on Structured Random Forest, Region Proposal Network and Rectangular Fitting

Abstract

ABSTRACT:In this paper we present a pipeline for high quality semantic segmentation of building facades using Structured Random Forest (SRF), Region Proposal Network (RPN) based on a Convolutional Neural Network (CNN) as well as rectangular fitting optimization. Our main contribution is that we employ features created by the RPN as channels in the SRF. We empirically show that this is very effective especially for doors and windows. Our pipeline is evaluated on two datasets where we outperform current state-of-the-art methods. Additionally, we quantify the contribution of the RPN and the rectangular fitting optimization on the accuracy of the result.

Facade Reconstruction for Textured Lod2 Citygml Models Based on Deep Learning and Mixed Integer Linear Programming

“…Thus, the false positive count is too high. Nevertheless, our average window segmentation accuracy of 0.94 exceeds by far the value of 0.84 of (Rahmani , Mayer, 2018).…”

“…Especially the field of facade parsing becomes increasingly popular for benchmarking new network architectures. Similar to our approach, in (Liu et al, 2017) Faster R-CNN and in (Rahmani , Mayer, 2018) an RPN is used for the segmentation of facades, see Section 1. Another improvement of Faster R-CNN was proposed in (He et al, 2017): "Masked R-CNN" is a combination of two already existing state-of-the-art models, an RPN and a binary mask classifier.…”

“…Recently, in (Rahmani , Mayer, 2018) a workflow combining a pre-trained RPN with a structured random forest approach (cf. Section 2) was introduced.…”

Comparison of Training Strategies for Convnets on Multiple Similar Datasets for Facade Segmentation

“…Table 4). Regarding the overall accuracy, on ECP data, even our worst result (0.93 with FT2) is better than the best result of the referred approaches (Rahmani and Mayer, 2018). All this leads us to the conclusion, that the limit of accuracy is almost reached for these datasets, due to the way the data was annotated.…”

“…By utilizing symmetries and repetitions their approach is capable to predict occluded facade objects. Rahmani et al (2017) used a Structured Random Forest (SRF) for pixel-wise labeling and extended their work in (Rahmani and Mayer, 2018) by adding proposals of a Region Proposal Network to the input of the SRF and by applying deterministic rectangular fitting. A sliding window detector, utilizing a cascade of weak classifiers, is proposed in (Neuhausen et al, 2018).…”