Automatic Annotation of Airborne Images by Label Propagation Based on a Bayesian-CRF Model

Abstract

The tremendous advances in deep neural networks have demonstrated the superiority of deep learning techniques for applications such as object recognition or image classification. Nevertheless, deep learning-based methods usually require a large amount of training data, which mainly comes from manual annotation and is quite labor-intensive. In order to reduce the amount of manual work required for generating enough training data, we hereby propose to leverage existing labeled data to generate image annotations automatically. Specifically, the pixel labels are firstly transferred from one image modality to another image modality via geometric transformation to create initial image annotations, and then additional information (e.g., height measurements) is incorporated for Bayesian inference to update the labeling beliefs. Finally, the updated label assignments are optimized with a fully connected conditional random field (CRF), yielding refined labeling for all pixels in the image. The proposed approach is tested on two different scenarios, i.e., (1) label propagation from annotated aerial imagery to unmanned aerial vehicle (UAV) imagery and (2) label propagation from map database to aerial imagery. In each scenario, the refined image labels are used as pseudo-ground truth data for training a convolutional neural network (CNN). Results demonstrate that our model is able to produce accurate label assignments even around complex object boundaries; besides, the generated image labels can be effectively leveraged for training CNNs and achieve comparable classification accuracy as manual image annotations, more specifically, the per-class classification accuracy of the networks trained by the manual image annotations and the generated image labels have a difference within ± 5 % .

A survey of image labelling for computer vision applications

“…Salisbury et al (2016) develop an ILS for realtime labelling of live aerial images by the crowd, aiming to rapidly identify points of interest and thus reducing the cognitive load of the pilots. Similarly, Zhuo et al (2019) propose an algorithm to simplify the annotation of airborne images. They use existing map data to transfer labels from a given ground truth to their own dataset.…”

“…Each labelling process starts with the material collection, which can be differentiated by its type (e.g., image or video) and further available properties such as meta data. If the current material does not suffice or is unevenly distributed, relevant images for model training can be sourced by crowdsourcing, by image scrapping from the Internet, or through the generation of artificial pictures (Zhuo et al, 2019). Subsequently, the actual labelling takes place as detailed in Section 4.…”

V-RSIR: An Open Access Web-Based Image Annotation Tool for Remote Sensing Image Retrieval

“…For example, in order to reduce the cost of labeling RS image, Yao et al [36] proposed a weakly supervised model with an efficient high-level semantic feature transferring scheme. Zhuo et al [37] automatically generated image annotations by label propagation based on a Bayesian-CRF model. To ensure their correctness, these automatically annotated images also should be pushed to volunteers for review.…”

Detection of seismic façade damages with multi-temporal oblique aerial imagery

“…For example, such CNN have been directly used in image classification (Hu et al 2015a(Hu et al , 2015bMaggiori et al 2017;Nogueira, Penatti, and Dos Santos 2017) and image segmentation (Kampffmeyer, Salberg, and Jenssen 2016;Längkvist et al 2016;Volpi and Tuia 2017) approaches. However, CNN have for example also been used to merge different modalities of remote sensing data (e.g., 3D and images) Lefèvre 2018, 2017;Duarte et al 2018a), annotate aerial images (Xia et al 2015;Zhuo et al 2019) and perform multi-temporal studies (Daudt et al 2018;Jung et al 2018;Wang et al 2018;Zhang et al 2019).…”