Semi-autonomous inspection for concrete structures using digital models and a hybrid approach based on deep learning and photogrammetry

Abstract

Bridge inspections are relied heavily on visual inspection, and usually conducted within limited time windows, typically at night, to minimize their impact on traffic. This makes it difficult to inspect every meter of the structure, especially for large-scale bridges with hard-to-access areas, which creates a risk of missing serious defects or even safety hazards. This paper presents a new technique for the semi-automated damage detection in tunnel linings and bridges using a hybrid approach based on photogrammetry and deep learning. The first approach involves using photogrammetry to reconstruct a 3D model. It is shown that a model with sub-centimeter accuracy can be obtained after noise removal. However, noise removal also reduces the point cloud density, making the 3D point cloud unsuitable for quantification of small-scale damages such as fine cracks. Therefore, the captured images are also analyzed using deep convolutional neural network (CNN) models to enable crack detection and segmentation. For this aim, in the second approach, the 3D model is generated by the output of CNN models to enable crack localization and quantification on 3D digital model. These two approaches were evaluated in separate case studies, showing that the proposed technique could be a valuable tool to assist human inspectors in detecting, localizing, and quantifying defects on concrete structures.

Automated reconstruction model of a cross‐sectional drawing from stereo photographs based on deep learning

“…Meanwhile, research in the construction field has also explored automated measurement using cameras, which boast significantly lower initial investment costs than TLS. Kardovskyi and Moon (2021) proposed a stereo camera and artificial intelligence-based model capable of measuring rebar spacing and length, achieving a measurement accuracy of ± 6 mm or less at a distance of 2 m. Mirzazade et al (2023) introduced a deep-learning-based semi-autonomous inspection technique that localized and quantified cracks inside tunnels by performing a 3D reconstruction of the tunnel interior using close-range photogrammetry, achieving sub-centimeter measurement accuracy at a distance of 2 m. In another study, Yamane et al (2023) proposed a method for detecting damage in bridges from images based on deep learning (DL) and structure from motion, recording the findings on a 3D bridge model to enable more efficient bridge inspection. However, while construction component dimension measurements using cameras currently require low initial investment costs, limitations exist in terms of measurement accuracy compared to methods utilizing TLS.…”

“…TLS is a device that employs light detection and ranging technology to measure the three‐dimensional (3D) point cloud data of target objects. High‐performance models in this product line are known to exhibit an impressive measurement accuracy of ± 5 mm at a distance of 100 m (Mirzazade et al., 2023). However, when applying the measured 3D point cloud data to construction component dimension measurement problems, post‐processing is required, which reduces the actual measurement accuracy.…”

“…Specifically, DL‐based CV methods hold the potential as a technological solution for implementing automated measurements in the construction field. DL‐based CV methods have been applied in the construction field, and in recent years, researchers have reported numerous achievements regarding automatic detection or segmentation of defects or cracks in construction components (Ge et al., 2023; Guo et al., 2021; Jiang & Zhang, 2020; Li et al., 2018; Malek et al., 2022; Mirzazade et al., 2023; Pan & Zhang, 2022; Yeum & Dyke, 2015). Despite the progress made, there are still challenges to overcome in implementing automated measurement in the construction field.…”

“…In particular, the dimensional quality of construction components is critical (Mirzaei et al., 2023), and therefore ensuring its control from the offsite production stage is recommended (Rausch et al., 2020; Q. Wang et al., 2017). However, measuring construction component dimensions is labor‐intensive and time‐consuming, resulting in high costs and the potential for human errors (Li et al., 2018; Mirzazade et al., 2023; Pan & Zhang, 2022; Q. Wang et al., 2017; Yeum & Dyke, 2015). Consequently, numerous studies have been conducted in the construction field to implement automated measurements.…”

“…As a result, it is not easily applied in fields with limited training datasets that are relatively difficult to obtain (Liu & Xu, 2022), such as the construction field. Consequently, most research on CV methods in the construction field has focused on crack detection (Ge et al., 2023; Jiang & Zhang, 2020; Li et al., 2018; Malek et al., 2022; Mirzazade et al., 2023; Yeum & Dyke, 2015), for which datasets are comparatively easier to acquire.…”