Nazr-CNN: Fine-Grained Classification of UAV Imagery for Damage Assessment

Abstract

We propose Nazr-CNN 1 , a deep learning pipeline for object detection and fine-grained classification in images acquired from Unmanned Aerial Vehicles (UAVs) for damage assessment and monitoring. Nazr-CNN consists of two components. The function of the first component is to localize objects (e.g. houses or infrastructure) in an image by carrying out a pixel-level classification. In the second component, a hidden layer of a Convolutional Neural Network (CNN) is used to encode Fisher Vectors (FV) of the segments generated from the first component in order to help discriminate between different levels of damage.To showcase our approach we use data from UAVs that were deployed to assess the level of damage in the aftermath of a devastating cyclone that hit the island of Vanuatu in 2015. The collected images were labeled by a crowdsourcing effort and the labeling categories consisted of fine-grained levels of damage to built structures. Since our data set is relatively small, a pretrained network for pixel-level classification and FV encoding was used. Nazr-CNN attains promising results both for object detection and damage assessment suggesting that the integrated pipeline is robust in the face of small data sets and labeling errors by annotators. While the focus of Nazr-CNN is on assessment of UAV images in a post-disaster scenario, our solution is general and can be applied in many diverse settings. We show one such case of transfer learning to assess the level of damage in aerial images collected after a typhoon in Philippines. 1 Nazr means "sight" in Arabic. 2

Detection of Disaster-Affected Cultural Heritage Sites from Social Media Images Using Deep Learning Techniques

“…Similarly, Pesaresi et al investigated rapid damage assessment of built-up structures using satellite data in tsunami-affected areas [74]. In order to produce comprehensive per-building damage scores, Fernandez Galarreta et al studied UAV-based urban structural damage assessment using object-based image analysis and semantic reasoning [26] whereas Attari et al explored fine-grained segmentation of UAV imagery based on deep learning techniques for damage assessment [8]. Alternatively, Vetrivel et al combined multiple kernel learning with 3D point cloud features derived from high resolution oblique aerial images to detect disaster damage [92].…”

Natural disasters detection in social media and satellite imagery: a survey

“…In [73], an existing deep model [112], pre-trained on Ima-geNet [43], is fine-tuned on aerial photos captured through unmanned aerial vehicles (UAV) during or after different types of natural disasters, namely floods, fires and building collapsed. Another work aiming damage assessment of natural disasters in images taken through UAV has been proposed by Nazr et al [15]. The adopted network is composed of two components.…”

Fine-Grained Land Use Classification at the City Scale Using Ground-Level Images

“…However, when we turn to fine-grained land use classification on large-scale instead of these toy examples, we could not avoid these challenges. There also exists some work [38], [39], [40] focusing on the concept of "fine-grained" in land use classification. However, [38] indicates fine granularity on time scale and [39] indicates fine granularity on levels of damage.…”