Semantic segmentation of point clouds of building interiors with deep learning: Augmenting training datasets with synthetic BIM-based point clouds

Won, Jong Uk; Czerniawski, Thomas; Leite, Fernanda

doi:10.1016/j.autcon.2020.103144

Cited by 127 publications

(53 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In the last decade various methods based on point clouds have been proposed, aiming to solve semantic segmentation. Semantic segmentation [ 10 , 11 , 12 ] can be broadly defined as the task of grouping parts of the input data, which can be 2D or 3D images or even 3D point clouds, which belong to the same object class, thus classifying each pixel or 3D point in the input according to a category.…”

Section: Introductionmentioning

confidence: 99%

“…Semantic segmentation was inspired by the success of Deep Learning methods in producing an accurate result [ 10 , 13 , 14 ], but these techniques require an extremely large amount of data to train the network. Such large datasets may be difficult to obtain, or not provide adequate information, such as the case of man-made structures captured by sensors that only provide colourless point clouds.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Underwater Object Recognition Using Point-Features, Bayesian Estimation and Semantic Information

Himri

Ridao

Gracias

2021

Sensors

View full text Add to dashboard Cite

This paper proposes a 3D object recognition method for non-coloured point clouds using point features. The method is intended for application scenarios such as Inspection, Maintenance and Repair (IMR) of industrial sub-sea structures composed of pipes and connecting objects (such as valves, elbows and R-Tee connectors). The recognition algorithm uses a database of partial views of the objects, stored as point clouds, which is available a priori. The recognition pipeline has 5 stages: (1) Plane segmentation, (2) Pipe detection, (3) Semantic Object-segmentation and detection, (4) Feature based Object Recognition and (5) Bayesian estimation. To apply the Bayesian estimation, an object tracking method based on a new Interdistance Joint Compatibility Branch and Bound (IJCBB) algorithm is proposed. The paper studies the recognition performance depending on: (1) the point feature descriptor used, (2) the use (or not) of Bayesian estimation and (3) the inclusion of semantic information about the objects connections. The methods are tested using an experimental dataset containing laser scans and Autonomous Underwater Vehicle (AUV) navigation data. The best results are obtained using the Clustered Viewpoint Feature Histogram (CVFH) descriptor, achieving recognition rates of 51.2%, 68.6% and 90%, respectively, clearly showing the advantages of using the Bayesian estimation (18% increase) and the inclusion of semantic information (21% further increase).

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Underwater Object Recognition Using Point-Features, Bayesian Estimation and Semantic Information

Himri

Ridao

Gracias

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…The methods that include region growingbased methods [20], [21], model fitting-based methods [22] and feature clustering-based methods [23] are usually adopted in this process. Then these elements are classified by rule-based methods [24]- [26], machine learning methods [27], [28] or deep learning methods [29], [30]. The above steps are not necessary for single object modeling, such as precast concrete elements [31].…”

Section: A Creation Of Bim Model From Point Cloudmentioning

confidence: 99%

A Survey of Applications With Combined BIM and 3D Laser Scanning in the Life Cycle of Buildings

Liu

Hyyppä

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The integration of building information model (BIM) with point cloud from 3D laser scanner provides powerful assistance in a variety of applications in the life cycle of building. However, the adoption of BIM and 3D laser scanning for applications in the whole life cycle of buildings has yet to be satisfactorily reviewed. There is a need to understand the current situation of applications with BIM and 3D laser scanning and their combined methods. The authors summarized combined methods of BIM and 3D laser scanning at first. Then the work presented in this paper provides summarization of applications in the life cycle of building. In addition, the impact of emerging technologies on applications also be discussed. In this review, articles were limited to those published over the last decade. Two patterns in the approach to integrate BIM and 3D laser scanning are reviewed. Several domains of application namely construction progress tracking, building components quality control, construction site safety, structural health monitoring, rescue after the disaster, energy modeling and management and modeling for exist building are looked back. According to occurred time of these applications, this paper divides them into two classes: applications in construction period and applications in maintenance period. Based on the observed limitations in the reviewed papers, the authors conclude some potential future research trend for these applications based emerging technology at the same time.

show abstract

“…In fact, a wide range of approaches exists for automatically modeling building interiors from point clouds. Obtaining a 3D interior model has been found quite challenging due to the presence of noise, occlusion, and clutter [1][2][3][4][5][6][7][8][9][10]. An important step in any indoor scene reconstruction method is the extraction of permanent structures (such as walls, floors, and ceilings) as they define the layout of the rooms that have to be reconstructed from the scanned point cloud.…”

Section: Introductionmentioning

confidence: 99%

“…They focused on manually designed features, whereas we will focus in this work on deep learning approaches, where the features are also learned automatically. Ma et al developed a method to extend a small dataset of captured data with synthetically generated point clouds from BIM models [10]. Using this approach, they improved results with deep learning methods for semantic segmentation.…”

Section: Introductionmentioning

confidence: 99%

Semantic Extraction of Permanent Structures for the Reconstruction of Building Interiors from Point Clouds

Coudron

Puttemans²,

Goedemé

et al. 2020

Sensors

View full text Add to dashboard Cite

The extraction of permanent structures (such as walls, floors, and ceilings) is an important step in the reconstruction of building interiors from point clouds. These permanent structures are, in general, assumed to be planar. However, point clouds from building interiors often also contain clutter with planar surfaces such as furniture, cabinets, etc. Hence, not all planar surfaces that are extracted belong to permanent structures. This is undesirable as it can result in geometric errors in the reconstruction. Therefore, it is important that reconstruction methods can correctly detect and extract all permanent structures even in the presence of such clutter. We propose to perform semantic scene completion using deep learning, prior to the extraction of permanent structures to improve the reconstruction results. For this, we started from the ScanComplete network proposed by Dai et al. We adapted the network to use a different input representation to eliminate the need for scanning trajectory information as this is not always available. Furthermore, we optimized the architecture to make inference and training significantly faster. To further improve the results of the network, we created a more realistic dataset based on real-life scans from building interiors. The experimental results show that our approach significantly improves the extraction of the permanent structures from both synthetically generated as well as real-life point clouds, thereby improving the overall reconstruction results.

show abstract

Semantic segmentation of point clouds of building interiors with deep learning: Augmenting training datasets with synthetic BIM-based point clouds

Cited by 127 publications

References 26 publications

Underwater Object Recognition Using Point-Features, Bayesian Estimation and Semantic Information

Underwater Object Recognition Using Point-Features, Bayesian Estimation and Semantic Information

A Survey of Applications With Combined BIM and 3D Laser Scanning in the Life Cycle of Buildings

Semantic Extraction of Permanent Structures for the Reconstruction of Building Interiors from Point Clouds

Contact Info

Product

Resources

About