Multi-Resolution Feature Fusion for Image Classification of Building Damages with Convolutional Neural Networks

Duarte, Diogo; Nex, Francesco; Kerle, N.; Vosselman, G.

doi:10.3390/rs10101636

Cited by 89 publications

(110 citation statements)

References 52 publications

Supporting

Mentioning

109

Contrasting

Order By: Relevance

“…In addition, machine learning algorithms are increasingly being used for data and image analysis [52,62,[72][73][74][75][76][77][78][79]. The CNN applied in the current study was tested to classify single black locust images under varying conditions and attained a high test accuracy of 99.5%.…”

Section: Discussionmentioning

confidence: 98%

“…Furthermore, there is an increasing interest in machine learning algorithms for data and image analysis, such as the application of the random forest model [52,[72][73][74][75][76], support vector machine [73][74][75][76], and deep learning algorithms, especially convolutional neural networks (CNNs) [62,73,75,[77][78][79]. However, CNNs were not previously utilized for the classification of black locust in short rotation coppices under varying conditions in single images.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robinia pseudoacacia L. in Short Rotation Coppice: Seed and Stump Shoot Reproduction as well as UAS-based Spreading Analysis

Carl

Lehmann²,

Landgraf

et al. 2019

Forests

View full text Add to dashboard Cite

Varying reproduction strategies are an important trait that tree species need in order both to survive and to spread. Black locust is able to reproduce via seeds, stump shoots, and root suckers. However, little research has been conducted on the reproduction and spreading of black locust in short rotation coppices. This research study focused on seed germination, stump shoot resprout, and spreading by root suckering of black locust in ten short rotation coppices in Germany. Seed experiments and sample plots were analyzed for the study. Spreading was detected and measured with unmanned aerial system (UAS)-based images and classification technology-object-based image analysis (OBIA). Additionally, the classification of single UAS images was tested by applying a convolutional neural network (CNN), a deep learning model. The analyses showed that seed germination increases with increasing warm-cold variety and scarification. Moreover, it was found that the number of shoots per stump decreases as shoot age increases. Furthermore, spreading increases with greater light availability and decreasing tillage. The OBIA and CNN image analysis technologies achieved 97% and 99.5% accuracy for black locust classification in UAS images. All in all, the three reproduction strategies of black locust in short rotation coppices differ with regards to initialization, intensity, and growth performance, but all play a role in the survival and spreading of black locust.Forests 2019, 10, 235 2 of 23 energy policies aimed towards a reduction in the use of fossil fuels and greater utilization of renewable energy since 2004 [7] marked an increase in fast-growing tree production, including black locust in short rotation coppices (harvested every 2-20 years) [8,9]. Nevertheless, some studies have declared black locust to be an invasive non-native tree species in Europe [6,[10][11][12]. However, other studies consider black locust to be an alternative tree species, for example for European ash (Fraxinus excelsior) [13,14], whose high dieback is caused by the fungal pathogen Hymenoscyphus fraxineus [14].Black locust possesses many growth characteristics that makes it ideal for short-rotation biomass production such as: rapid growth, high drought tolerance, and nitrogen-fixation [15,16], as well as the ability to reproduce via stump shoots in response to harvest. Black locust starts flowering at the age of six years [17]. Many insects benefit from this characteristic, including honeybees [18]. Moreover, the production of black locust honey is very common and economically important, especially in Hungary [17]. A maximum of 15,559 flowers per tree were counted in an eight-year-old black locust plantation [18], and 12,000 seeds/m 2 were identified in a monodominant stand [6,19]; black locust thus produces an abundance of seeds. As black locust belongs to the Fabaceae family, the seed coat is hard and impermeable [20,21]. Hence, seeds require scarification for successful germination [21][22][23] and priming of seeds is favorable [21,24,25]. Seeds pre...

show abstract

Section: Discussionmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

Robinia pseudoacacia L. in Short Rotation Coppice: Seed and Stump Shoot Reproduction as well as UAS-based Spreading Analysis

Carl

Lehmann²,

Landgraf

et al. 2019

Forests

View full text Add to dashboard Cite

show abstract

“…Remote sensing has been continuously used for automatic building damage assessment, since the used platforms can cover large areas and attenuate the costly and lengthy ground observations. While several sensors coupled with distinct platforms have been used (Armesto-González et al, 2010;Dell'Acqua and Polli, 2011;Gokon et al, 2015), there has been a special interest in the use of images (Duarte et al, 2018a;Tu et al, 2017;Vetrivel et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

Damage Detection on Building Façades Using Multi-Temporal Aerial Oblique Imagery

Duarte

Nex

Kerle

et al. 2019

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

Self Cite

View full text Add to dashboard Cite

<p><strong>Abstract.</strong> Over the past decades, a special interest has been given to remote-sensing imagery to automate the detection of damaged buildings. Given the large areas it may cover and the possibility of automation of the damage detection process, when comparing with lengthy and costly ground observations. Currently, most image-based damage detection approaches rely on Convolutional Neural Networks (CNN). These are used to determine if a given image patch shows damage or not in a binary classification approach. However, such approaches are often trained using image samples containing only debris and rubble piles. Since such approaches often aim at detecting partial or totally collapsed buildings from remote-sensing imagery. Hence, such approaches might not be applicable when the aim is to detect façade damages. This is due to the fact that façade damages also include spalling, cracks and other small signs of damage. Only a few studies focus their damage analysis on the façade and a multi-temporal approach is still missing. In this paper, a multi-temporal approach specifically designed for the image classification of façade damages is presented. To this end, three multi-temporal approaches are compared with two mono-temporal approaches. Regarding the multi-temporal approaches the objective is to understand the optimal fusion between the two imagery epochs within a CNN. The results show that the multi-temporal approaches outperform the mono-temporal ones by up to 22% in accuracy.</p>

show abstract

“…Convolutional neural networks (CNN) was utilized to identify collapsed buildings from post-event satellite imagery and obtained an OA of 80.1% and Kappa of 0.46 [17]. Multiresolution feature maps were derived and fused with CNN for the image classification of building damages in [18], and an OA of 88.7% was obtained.Most of the above-mentioned damage information extraction studies classified damaged buildings into two classes: damaged and intact. However, these two classes are not enough to meet actual needs.Recently, deep learning (DL) methods have provided new ideas for remote sensing image recognition technology.…”

mentioning

confidence: 99%

mentioning

confidence: 99%

Assessment of the Degree of Building Damage Caused by Disaster Using Convolutional Neural Networks in Combination with Ordinal Regression

Liu

Wang

2019

Remote Sensing

View full text Add to dashboard Cite

We propose a new convolutional neural networks method in combination with ordinal regression aiming at assessing the degree of building damage caused by earthquakes with aerial imagery. The ordinal regression model and a deep learning algorithm are incorporated to make full use of the information to improve the accuracy of the assessment. A new loss function was introduced in this paper to combine convolutional neural networks and ordinal regression. Assessing the level of damage to buildings can be considered as equivalent to predicting the ordered labels of buildings to be assessed. In the existing research, the problem has usually been simplified as a problem of pure classification to be further studied and discussed, which ignores the ordinal relationship between different levels of damage, resulting in a waste of information. Data accumulated throughout history are used to build network models for assessing the level of damage, and models for assessing levels of damage to buildings based on deep learning are described in detail, including model construction, implementation methods, and the selection of hyperparameters, and verification is conducted by experiments. When categorizing the damage to buildings into four types, we apply the method proposed in this paper to aerial images acquired from the 2014 Ludian earthquake and achieve an overall accuracy of 77.39%; when categorizing damage to buildings into two types, the overall accuracy of the model is 93.95%, exceeding such values in similar types of theories and methods. a set of 13 change detention features and support vector machine (SVM). Simon Plank [12] reviewed the methods of rapid damage assessment using multitemporal Synthetic Aperture Radar(SAR) data. Gupta et al. [13] present a satellite imagery dataset for building damage assessment with over 700,000 labeled building instances covering over 5000 km 2 of imagery.Recent studies show that the machine learning algorithm performs well in earthquake damage assessment. Li [14] assessed building damage with one-class SVM using pre-and post-earthquake QuickBird imagery and assessed the discrimination power of different level (pixel-level, texture, and object-based) features. Haiyang et al. [15] combined SVM and the image segmentation method to detect building damage. Cooner et al. [16] evaluate the effectiveness of machine learning algorithms in detecting earthquake damage. A series of textural and structural features were used in this study. A SVM and feature selection approach was carried out for damage mapping with post-event very high spatial resolution(VHR) image and obtained overall accuracy (OA) of 96.8% and Kappa of 0.5240 [11]. Convolutional neural networks (CNN) was utilized to identify collapsed buildings from post-event satellite imagery and obtained an OA of 80.1% and Kappa of 0.46 [17]. Multiresolution feature maps were derived and fused with CNN for the image classification of building damages in [18], and an OA of 88.7% was obtained.Most of the above-mentioned damage information extracti...

show abstract

Multi-Resolution Feature Fusion for Image Classification of Building Damages with Convolutional Neural Networks

Cited by 89 publications

References 52 publications

Robinia pseudoacacia L. in Short Rotation Coppice: Seed and Stump Shoot Reproduction as well as UAS-based Spreading Analysis

Robinia pseudoacacia L. in Short Rotation Coppice: Seed and Stump Shoot Reproduction as well as UAS-based Spreading Analysis

Damage Detection on Building Façades Using Multi-Temporal Aerial Oblique Imagery

Assessment of the Degree of Building Damage Caused by Disaster Using Convolutional Neural Networks in Combination with Ordinal Regression

Contact Info

Product

Resources

About