Visual data classification in post-event building reconnaissance

Yeum, Chul Min; Dyke, Shirley J.; Ramírez, Julio A.

doi:10.1016/j.engstruct.2017.10.057

Cited by 105 publications

(62 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In other words, 78.8% of all damages annotated as Damage 1 was correctly predicted on the average, and likewise for other classes and CNN architectures. The said precision and recall values are considerably higher in comparison with that reported by Yeum et al for single class (spalling) detection (Precision: 40.48%, Recall: 62.16 %) on similar dataset. Minor cracks in concrete are typically hard to detect due to potential noise infusion .…”

Section: Resultscontrasting

confidence: 56%

Deep learning‐based multi‐class damage detection for autonomous post‐disaster reconnaissance

Mondal

Jahanshahi

et al. 2020

Struct Control Health Monit

View full text Add to dashboard Cite

Timely assessment of damages induced to buildings due to an earthquake is critical for ensuring life safety, mitigating financial losses, and expediting the rehabilitation process as well as enhancing the structural resilience where resilience is measured by an infrastructure's capacity to restore full functionality post extreme events. Since manual inspection is expensive, time consuming and risky, low‐cost unmanned aerial vehicles or robots can be leveraged as a viable alternative for quick reconnaissance. Visual data captured by the sensors mounted on the robots can be analyzed, and the damages can be detected and classified autonomously. The present study proposes the use of deep learning‐based approaches to this end. Region‐based convolutional neural network (Faster RCNN) is exploited to detect four different damage types, namely, surface crack, spalling (which includes façade spalling and concrete spalling), and severe damage with exposed rebars and severely buckled rebars. The performance of the proposed approach is evaluated on manually annotated image data collected from reinforced concrete buildings damaged under several past earthquakes such as Nepal (2015), Taiwan (2016), Ecuador (2016), Erzincan (1992), Duzce (1999), Bingol (2003), Peru (2007), Wenchuan (2008), and Haiti (2010). Several experiments are presented in the paper to illustrate the capabilities, as well as the limitations, of the proposed approach for earthquake reconnaissance. It was observed that Inception‐ResNet‐v2 significantly outperforms the other networks considered in this study. The research outcome is a stepping stone forward to facilitate the autonomous assessment of buildings where this can be potentially useful for insurance companies, government agencies, and property owners.

show abstract

Section: Resultscontrasting

confidence: 56%

Deep learning‐based multi‐class damage detection for autonomous post‐disaster reconnaissance

Mondal

Jahanshahi

et al. 2020

Struct Control Health Monit

View full text Add to dashboard Cite

show abstract

“…The convolution kernels in DCNN capture the spatial invariant characteristics such as edges and contrast from the input image where these features are then used to make inference about the image. Since 2017, the rapid growth of DCNNbased approaches for damage detection in civil engineering has shown huge potential (Atha & Jahanshahi, 2018;Cha, Choi, & Büyüköztürk, 2017;Cha, Choi, Suh, Mahmoudkhani, & Büyüköztürk, 2018;Chen & Jahanshahi, 2018;Gao & Mosalam, 2018;Kumar, Abraham, Jahanshahi, Iseley, & Starr, 2018;Lin, Nie, & Ma, 2017;Yeum, Dyke, Ramirez, & Benes, 2016;Yeum, Dyke, & Ramirez, 2018;Wu & Jahanshahi, 2018b;Xue & Li, 2018;Zhang et al, 2017). However, the high computation and memory demands required for DCNN make it inappropriate for deployment on mobile inspection devices, such as unmanned aerial vehicles (UAVs) and robots.…”

Section: Background and Motivationmentioning

confidence: 99%

Pruning deep convolutional neural networks for efficient edge computing in condition assessment of infrastructures

Singla

Jahanshahi

et al. 2019

Computer aided Civil Eng

View full text Add to dashboard Cite

Health monitoring of civil infrastructures is a key application of Internet of things (IoT), while edge computing is an important component of IoT. In this context, swarms of autonomous inspection robots, which can replace current manual inspections, are examples of edge devices. Incorporation of pretrained deep learning algorithms into these robots for autonomous damage detection is a challenging problem since these devices are typically limited in computing and memory resources. This study introduces a solution based on network pruning using Taylor expansion to utilize pretrained deep convolutional neural networks for efficient edge computing and incorporation into inspection robots. Results from comprehensive experiments on two pretrained networks (i.e., VGG16 and ResNet18) and two types of prevalent surface defects (i.e., crack and corrosion) are presented and discussed in detail with respect to performance, memory demands, and the inference time for damage detection. It is shown that the proposed approach significantly enhances resource efficiency without decreasing damage detection performance.

show abstract

“…For instance, if the building images used for training only contain wooden buildings, the classifier may not be sufficiently accurate when classifying images of masonry or concrete buildings (Yeum, Dyke, & Ramirez, 2018). This building classifier is trained in advance using a large volume of ground-truth building images.…”

Section: Algorithmmentioning

confidence: 99%

“…The ground-truth building images used for training the classifier must include images of buildings that have a similar appearance as the target building. For instance, if the building images used for training only contain wooden buildings, the classifier may not be sufficiently accurate when classifying images of masonry or concrete buildings (Yeum, Dyke, & Ramirez, 2018). After applying the classifier to ⟂ 0 , we determine a tight bounding box for the building in that image.…”

Section: Algorithmmentioning

confidence: 99%

Automated building image extraction from 360° panoramas for postdisaster evaluation

Lenjani

Yeum

Dyke

et al. 2019

Computer aided Civil Eng

Self Cite

View full text Add to dashboard Cite

After a disaster, teams of structural engineers collect vast amounts of images from damaged buildings to obtain new knowledge and extract lessons from the event. However, in many cases, the images collected are captured without sufficient spatial context. When damage is severe, it may be quite difficult to even recognize the building. Accessing images of the predisaster condition of those buildings is required to accurately identify the cause of the failure or the actual loss in the building. Here, to address this issue, we develop a method to automatically extract pre‐event building images from 360° panorama images (panoramas). By providing a geotagged image collected near the target building as the input, panoramas close to the input image location are automatically downloaded through street view services (e.g., Google or Bing in the United States). By computing the geometric relationship between the panoramas and the target building, the most suitable projection direction for each panorama is identified to generate high‐quality 2D images of the building. Region‐based convolutional neural networks are exploited to recognize the building within those 2D images. Several panoramas are used so that the detected building images provide various viewpoints of the building. To demonstrate the capability of the technique, we consider residential buildings in Holiday Beach in Rockport, Texas, United States, that experienced significant devastation in Hurricane Harvey in 2017. Using geotagged images gathered during actual postdisaster building reconnaissance missions, we verify the method by successfully extracting residential building images from Google Street View images, which were captured before the event.

show abstract

Visual data classification in post-event building reconnaissance

Cited by 105 publications

References 21 publications

Deep learning‐based multi‐class damage detection for autonomous post‐disaster reconnaissance

Deep learning‐based multi‐class damage detection for autonomous post‐disaster reconnaissance

Pruning deep convolutional neural networks for efficient edge computing in condition assessment of infrastructures

Automated building image extraction from 360° panoramas for postdisaster evaluation

Contact Info

Product

Resources

About