Vision-based real-time structural vibration measurement through deep-learning-based detection and tracking methods

Three‐dimensional (3D) object detection, that is, localizing and classifying all critical objects in a 3D space, is essential for downstream construction scene analysis tasks. However, accurate instance segmentation, few 2D object segmentation and 3D object detection data sets, high‐quality feature representations for depth estimation, and limited 3D cues from a single red‐green‐blue (RGB) image pose significant challenges to 3D object detection and severely hinder its practical applications. In response to these challenges, an improved cascade‐based network with a transformer backbone and a boundary‐patch‐refinement method is proposed to build hierarchical features and refine object boundaries, resulting in better results in 2D object detection and instance segmentation. Furthermore, a novel self‐supervised monocular depth learning method is proposed to extract better feature representations for depth estimation from construction site video data with unknown camera parameters. Additionally, a pseudo‐LiDAR point cloud method and a 3D object detection method with a density‐based clustering algorithm are proposed to detect 3D objects in a construction scene without help from 3D labels, which will serve as a good foundation for other downstream 3D tasks. Finally, the proposed model is evaluated for object instance segmentation and depth estimation on the moving objects in construction sites (MOCS) and construction scene data sets. It brings a 9.16% gain in terms of mean average precision (mAP) for object detection and a 4.92% gain in mask mAP for object instance segmentation. The average order accuracy and relative mean error for depth estimation are improved by 0.94% and 60.56%, respectively. This study aims to overcome the challenges and limitations of 3D object detection and facilitate practical applications in construction scene analysis.

Section: Discussionmentioning

confidence: 99%

Monocular 3D object detection for construction scene analysis

Shen,

Jiao,

Zhang

et al. 2023

“…Perez-Ramirez et al (2019) reported on the response analysis of a large building entailing the use of a recurrent neural network based on the fact that vibration signals are time-series data. Pan et al (2023) proposed a method to measure the magnitude of structural vibration through a vision-based algorithm. Furthermore, Rafiei et al (2017) proposed a deep learning algorithm to predict the compressive strength of concrete.…”

Section: Deep Learning In Civil Engineeringmentioning

confidence: 99%

“…Pan et al. (2023) proposed a method to measure the magnitude of structural vibration through a vision‐based algorithm. Furthermore, Rafiei et al.…”

Section: Literature Reviewmentioning

confidence: 99%

Self‐training approach for crack detection using synthesized crack images based on conditional generative adversarial network

Shim

2023

Urban infrastructure plays a crucial role in determining the quality of life for citizens. However, given the increasing number of aging infrastructures, regular inspections are essential to prevent accidents. Deep learning studies have been conducted to detect structural damage and ensure high accuracy and reliability of these inspections. However, these detection algorithms often face challenges due to scarcity of damage data. To overcome this issue, this paper proposes a method for synthesizing crack images and utilizing them for crack detection. Initially, crack images are synthesized from labeled images by using a conditional generative adversarial network. Subsequently, a new self‐training method is implemented wherein the synthesized crack images from the prediction images were incorporated into the learning process to enhance data diversity. The proposed approach yields promising results with a mean intersection over union of 80.34% and F1‐score of 76.31% on average. The proposed method can aid further research on virtual image generation for crack detection, seeking to reduce the reliance on extensive image collection.

“…However, adopting ML and DL techniques, monitoring and control systems become rapidly advanced. Pan et al (2023) applied deep CNNs for tracking targets in videos to monitor vibrations of multistory buildings. Jabadinasab-Hormozabad et al ( 2021) proposed a system integrating structural control, health monitoring, and energy harvesting.…”

Section: Literature Reviewmentioning

confidence: 99%

Road crack detection interpreting background images by convolutional neural networks and a self‐organizing map

Yamaguchi,

Mizutani

2023

The presence of road cracks is an important indicator of damage. Deep learning is a prevailing method for detecting cracks in road surface images because of its detection ability. Previous research works focused on supervised convolutional neural networks (CNNs) without non‐crack features or unsupervised crack analysis with limited accuracies. The novelty of this study is the addition of background classification. By increasing the number of non‐crack categories, CNNs are driven to learn non‐crack features and improve crack detection performances. Non‐crack images are preprocessed, and their features are extracted in an unsupervised way by a deep convolutional autoencoder. A self‐organizing map clusters features to obtain non‐crack categories. This study focusses on classification though the method can be adopted in parallel with the latest segmentation algorithms. Using common road crack datasets, modified deep CNN models significantly improved accuracy by 1%–4% and f‐measure by 3%–8%, compared to previous models. The modified visual geometry group (VGG) 16 showed the top‐level performance, 96% accuracy and 84%–85% f‐measure. The models drastically reduced false detection cases while maintaining their crack detection abilities.