Multi‐defect segmentation from façade images using balanced copy–paste method

Li, Jiajun; Wang, Qian; Ma, Jun; Guo, Jingjing

doi:10.1111/mice.12808

Cited by 18 publications

(7 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instance segmentation, however, is a task combining semantic segmentation and object detection, which not only classifies each pixel but differentiates different objects of the same class. Mask R-CNN ( 24 ) is a deep learning model of instance segmentation, and it has been successfully used for segmenting concrete bridge cracks ( 25 ), façade defects ( 26 ), and shield tunnel lining cracks ( 27 ).…”

Section: Literature Reviewmentioning

confidence: 99%

A Multitask Deep Learning Model for Parsing Bridge Elements and Segmenting Defect in Bridge Inspection Images

Zhang

Karim

Qin

2023

Transportation Research Record: Journal of the Transportation R

View full text Add to dashboard Cite

The vast network of bridges in the United States has a high requirement for maintenance and rehabilitation. The massive cost of manual visual inspection to assess bridge conditions is a burden to some extent. Advanced robots have been used to automate inspection data collection. Automating the segmentations of multiclass elements and surface defects on the elements in the large volume of inspection image data would facilitate an efficient and effective assessment of bridge conditions. Training separate single-task networks for element parsing (i.e., semantic segmentation of multiclass elements) and defect segmentation fails to incorporate the close connection between these two tasks. Both recognizable structural elements and apparent surface defects are present in the inspection images. This paper is motivated to develop a multitask deep learning model that fully uses such interdependence between bridge elements and defects to boost the model’s task performance and generalization. Furthermore, the study investigated the effectiveness of the proposed model designs for improving task performance, including feature decomposition, cross-talk sharing, and multi-objective loss function. A data set with pixel-level labels of bridge elements and corrosion was developed for model training and testing. Quantitative and qualitative results from evaluating the developed multitask deep model demonstrate its advantages over the single-task-based model not only in performance (2.59% higher mIoU on bridge parsing and 1.65% on corrosion segmentation) but also in computational time and implementation capability.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

A Multitask Deep Learning Model for Parsing Bridge Elements and Segmenting Defect in Bridge Inspection Images

Zhang

Karim

Qin

2023

Transportation Research Record: Journal of the Transportation R

View full text Add to dashboard Cite

show abstract

“…For those application areas that do not have a large open-sourced data set containing complete categories and various features, data set creation plays an important role during deep learning as the data set is required to be created by a series of time-consuming and sophisticated steps (Barbedo, 2018). Defects detection in buildings and infrastructure is such a typical application area (Li et al, 2019(Li et al, , 2022. Hence, there has been an increasing amount of literature on data quantity problem in recent years.…”

Section: Data Quantity Problem For Defects Detectionmentioning

confidence: 99%

Informativeness‐guided active learning for deep learning–based façade defects detection

Guo

Wang

et al. 2023

Computer aided Civil Eng

Self Cite

View full text Add to dashboard Cite

Annotation work is burdensome and challenging for developing a façade defects detector, especially when the raw data set is large but not all useful. To alleviate the problem, this study proposes an informativeness‐guided active learning methodology to effectively select informative data to train a robust façade defects detector. A novel data annotation workflow is developed to ensure the high quality of labels. Then, an active learning–based model training strategy is adopted to enable the model to have both the abilities of generalization and discrimination on different defect features. Besides, an innovative informativeness assessment method is proposed by flexibly combining the degree of uncertainty and the degree of representativeness. Through the proposed method, the performance of façade defects detection can be further boosted with the same amount but more informative training data so that the cost‐efficiency of human annotation work can be improved.

show abstract

“…Although the defect detection methods based on computer vision (CV) have gained approved performance (J. Li et al., 2022; Zhou et al., 2022), there are still several issues to be resolved. For instance, an excellent CV‐based approach depends on a feature extractor that can effectively extract deep and semantically strong features.…”

Section: Introductionmentioning

confidence: 99%

Attention‐guided multiscale neural network for defect detection in sewer pipelines

Wang

Dang

et al. 2023

Computer aided Civil Eng

View full text Add to dashboard Cite

Sanitary sewer systems are major infrastructures in every modern city, which are essential in protecting water pollution and preventing urban waterlogging. Since the conditions of sewer systems continuously deteriorate over time due to various defects and extrinsic factors, early intervention in the defects is necessary to prolong the service life of the pipelines. However, prior works for defect inspection are limited by accuracy, efficiency, and economic cost. In addition, the current loss functions in object detection approaches are unable to handle the imbalanced data well. To address the above drawbacks, this paper proposes an automatic defect detection framework that accurately identifies and localizes eight types of defects in closed‐circuit television videos based on a deep neural network. First, an effective attention module is introduced and used in the backbone of the detector for better feature extraction. Then, a novel feature fusion mechanism is presented in the neck to alleviate the problem of feature dilution. After that, an efficient loss function that can reasonably adjust the weight of training samples is proposed to tackle the imbalanced data problem. Also, a publicly available dataset is provided for defect detection tasks. The proposed detection framework is robust against the imbalanced data and achieves a state‐of‐the‐art mean average precision of 73.4%, which is potentially applied in realistic sewer defect inspections.

show abstract

Multi‐defect segmentation from façade images using balanced copy–paste method

Cited by 18 publications

References 52 publications

A Multitask Deep Learning Model for Parsing Bridge Elements and Segmenting Defect in Bridge Inspection Images

A Multitask Deep Learning Model for Parsing Bridge Elements and Segmenting Defect in Bridge Inspection Images

Informativeness‐guided active learning for deep learning–based façade defects detection

Attention‐guided multiscale neural network for defect detection in sewer pipelines

Contact Info

Product

Resources

About