Small Defect Instance Reconstruction Based on 2D Connectivity-3D Probabilistic Voting

Hong, Kunlong; Wang, Hongguang; Zhu, Bing

doi:10.1109/robio54168.2021.9739608

Cited by 3 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspection-Net [16] is used to segment spalling and cracking within a tunnel. Hong investigates a more suitable loss function for Inspection-Net [11]. As an advanced crack and spalling segmentation network, Inspection-Net achieves the best recognition accuracy on the CSSC dataset.…”

Section: Concrete Surface Defect Detection Based On Deep Learningmentioning

confidence: 99%

“…The instance number for each class starts at 1 and goes up to 256. We use our previous naive distance TSDF method [11] as the baseline, which uses the center distance threshold of different initial instances in a class to fuse them together. For our back-projection and adjacency matrix approach, the 3D IoU thresholds θ c in Equation ( 9) for crack, patched area, erosion, and spots were set to 0.02, 0.1, 0.1 and 0.01.…”

Section: Defect Surface Reconstruction Testmentioning

confidence: 99%

“…The dam's flood discharge spillways and drift spillways are also the focus of the maintenance procedure, in comparison to the long-term monitoring of the dam, because they frequently experience scouring by water flow. Our previous work used the SLAM approach to reconstruct the sidewall of drift spillways with crack flaws [11]. Dam surface monitoring needs to realize defects' segmentation, statistics, and reconstruction.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multiple Defects Inspection of Dam Spillway Surface Using Deep Learning and 3D Reconstruction Techniques

Hong

Wang²,

Yuan³

et al. 2023

Buildings

Self Cite

View full text Add to dashboard Cite

After a lengthy period of scouring, the reinforced concrete surface of the dam spillway (i.e., drift spillways and flood discharge spillways) will suffer from deterioration and damage. Regular manual inspection is time-consuming and dangerous. This paper presents a robotic solution to detect automatically, count defect instance numbers, and reconstruct the surface of dam spillways by incorporating the deep learning method with a visual 3D reconstruction method. The lack of a real dam defect dataset and incomplete registration of minor defects on the 3D mesh model in fusion step are two challenges addressed in the paper. We created a multi-class semantic segmentation dataset of 1711 images (with resolutions of 848 × 480 and 1280 × 720 pixels) acquired by a wall-climbing robot, including cracks, erosion, spots, patched areas, and power safety cable. Then, the architecture of the U-net is modified with pixel-adaptive convolution (PAC) and conditional random field (CRF) to segment different scales of defects, trained, validated, and tested using this dataset. The reconstruction and recovery of minor defect instances in the flow surface and sidewall are facilitated using a keyframe back-projection method. By generating an instance adjacency matrix within the class, the intersection over union (IoU) of 3D voxels is calculated to fuse multiple instances. Our segmentation model achieves an average IoU of 60% for five defect class. For the surface model’s semantic recovery and instance statistics, our method achieves accurate statistics of patched area and erosion instances in an environment of 200 m2, and the average absolute error of the number of spots and cracks has reduced from the original 13.5 to 3.5.

show abstract

Section: Concrete Surface Defect Detection Based On Deep Learningmentioning

confidence: 99%

Section: Defect Surface Reconstruction Testmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multiple Defects Inspection of Dam Spillway Surface Using Deep Learning and 3D Reconstruction Techniques

Hong

Wang²,

Yuan³

et al. 2023

Buildings

Self Cite

View full text Add to dashboard Cite

show abstract

“…At present, there are three main methods for indoor BIM reconstructions, namely, image-based 3D reconstructions [9], traditional geometric modeling techniques [10], and point cloud-based 3D reconstruction methods [11]. The image-based 3D reconstruction method requires a large amount of image data from different perspectives [12].…”

Section: Introductionmentioning

confidence: 99%

Automated Reconstruction of Existing Building Interior Scene BIMs Using a Feature-Enhanced Point Transformer and an Octree

Chen,

Liang,

Xie

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Building information models (BIMs) offer advantages, such as visualization and collaboration, making them widely used in the management of existing buildings. Currently, most BIMs for existing indoor spaces are manually created, consuming a significant amount of manpower and time, severely impacting the efficiency of building operations and maintenance management. To address this issue, this study proposes an automated reconstruction method for an indoor scene BIM based on a feature-enhanced point transformer and an octree. This method enhances the semantic segmentation performance of point clouds by using feature position encoding to strengthen the point transformer network. Subsequently, the data are partitioned into multiple segments using an octree, collecting the geometric and spatial information of individual objects in the indoor scene. Finally, the BIM is automatically reconstructed using Dynamo in Revit. The research results indicate that the proposed feature-enhanced point transformer algorithm achieves a high segmentation accuracy of 71.3% mIoU on the S3DIS dataset. The BIM automatically generated from the field point cloud data, when compared to the original data, has an average error of ±1.276 mm, demonstrating a good reconstruction quality. This method achieves the high-precision, automated reconstruction of the indoor BIM for existing buildings, avoiding extensive manual operations and promoting the application of BIMs for the maintenance processes of existing buildings.

show abstract

Inspection-Nerf: Rendering Multi-Type Local Images for Dam Surface Inspection Task Using Climbing Robot and Neural Radiance Field

2023

View full text Add to dashboard Cite

For the surface defects inspection task, operators need to check the defect in local detail images by specifying the location, which only the global 3D model reconstruction can’t satisfy. We explore how to address multi-type (original image, semantic image, and depth image) local detail image synthesis and environment data storage by introducing the advanced neural radiance field (Nerf) method. We use a wall-climbing robot to collect surface RGB-D images, generate the 3D global model and its bounding box, and make the bounding box correspond to the Nerf implicit bound. After this, we proposed the Inspection-Nerf model to make Nerf more suitable for our near view and big surface scene. Our model use hash to encode 3D position and two separate branches to render semantic and color images. And combine the two branches’ sigma values as density to render depth images. Experiments show that our model can render high-quality multi-type images at testing viewpoints. The average peak signal-to-noise ratio (PSNR) equals 33.99, and the average depth error in a limited range (2.5 m) equals 0.027 m. Only labeled 2% images of 2568 collected images, our model can generate semantic masks for all images with 0.957 average recall. It can also compensate for the difficulty of manual labeling through multi-frame fusion. Our model size is 388 MB and can synthesize original and depth images of trajectory viewpoints within about 200 m2 dam surface range and extra defect semantic masks.

show abstract

Small Defect Instance Reconstruction Based on 2D Connectivity-3D Probabilistic Voting

Cited by 3 publications

References 13 publications

Multiple Defects Inspection of Dam Spillway Surface Using Deep Learning and 3D Reconstruction Techniques

Multiple Defects Inspection of Dam Spillway Surface Using Deep Learning and 3D Reconstruction Techniques

Automated Reconstruction of Existing Building Interior Scene BIMs Using a Feature-Enhanced Point Transformer and an Octree

Inspection-Nerf: Rendering Multi-Type Local Images for Dam Surface Inspection Task Using Climbing Robot and Neural Radiance Field

Contact Info

Product

Resources

About