Multilabel Remote Sensing Image Annotation With Multiscale Attention and Label Correlation

Abstract

Deep learning based multi-label image annotation is receiving increasing attention in the field of remote sensing due to the great success of deep networks in single-label remote sensing image classification. Compared with those low-level features, the features extracted by the convolutional neural network (CNN) are more informative and can alleviate the problem of semantic gap. However, the CNN model tends to ignore the smaller objects when objects of different sizes exist in an image. In addition, how to efficiently leverage the correlation among multiple labels to enhance annotation performance remains an open issue. In this paper, we propose an end-to-end deep learning framework for multi-label remote sensing image annotation. The framework is composed of a multi-scale feature fusion module, a channelspatial attention learning module and a label correlation extraction module. The multi-scale features from different layers of a CNN model are first fused and refined by using a channel-spatial attention mechanism. Then, the label correlation information is extracted from a label co-occurrence matrix, and embedded into the multi-scale attentive features to increase the discriminative ability of the resulting image features. The experiments on two benchmark data sets demonstrate the superiority of the proposed method in comparison with the state-of-the-art methods.

A Confounder-Free Fusion Network for Aerial Image Scene Feature Representation

“…The conventional feature representation methods are based on low-level visual features [41,42], which are not discriminative enough to represent complex image content. Recently, many learning-based methods [43][44][45][46] have achieved great success by representing high-level visual features of images. For instance, Xiong et al [4] utilize the attention mechanism and multi-task learning strategy to improve the discriminant capabilities of the model.…”

“…To preserve the scene-level similarity relationships of images, attention mechanism and skip-layer connection strategy are combined to produce discriminative features for object-level aerial images annotation [45]. Huang et al [46] fuse the multiscale features from different layers and hence introduce an attention model to increase the discriminative ability of feature representation. However, it is evident that these methods fail to extract the features with different grained levels.…”

Patch-Based Discriminative Learning for Remote Sensing Scene Classification

“…The authors trained their model in an end-to-end manner where CNN layers are formed to generate mid-level features and RNN is used for learning contextual dependencies. Huang et al [36] proposed an end-to-end deep learning model with multiscale feature fusion, channel-spatial attention, and a label correlation extraction module. Specifically, a channel-spatial attention mechanism is used to fuse and refine multi-scale features from different layers of the CNN model.…”

Automatic Image Annotation Using Adaptive Convolutional Deep Learning Model

“…Then it creates a sense for the assorted area of the pixels, which is really what the image segment wants to get. In this way, order is paramount during clinical picture commentary [4]. Therefore, in recent years, many researchers in this field have made relentless efforts to further improve the accuracy of the sequence, although the characterization of the images concept problem is indeed critical of many problems and difficulties [5].…”