Multi-scale object detection in remote sensing imagery with convolutional neural networks

Deng, Zhipeng; Sun, Hao; Zhou, Shilin; Zhao, Jiewen; Lei, Lin; Zou, Huanxin

doi:10.1016/j.isprsjprs.2018.04.003

Cited by 388 publications

(168 citation statements)

References 51 publications

Supporting

Mentioning

167

Contrasting

Order By: Relevance

“…where 11 Conv  and 33 Conv  indicate a 11  convolution layer and a 33  convolution layer, respectively; Upsample indicates bilinear up-sampling and then performs a 11  convolution;…”

Section: Backbone Network and Rpnmentioning

confidence: 99%

“…Wei et al [10] came up with a HR ship detection network (HR-SDNet) to perform precise and robust ship detection in SAR images. Deng et al [11] devised a method to detect multiscale artificial targets in remote sensing images. An et al [12] came up with a DRBox-v2 with rotatable boxes to boost the precision and recall rates of detection for object detection in HR SAR images.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

HQ-ISNet: High-Quality Instance Segmentation for Remote Sensing Imagery

Wei

Liu

et al. 2020

Remote Sensing

View full text Add to dashboard Cite

Instance segmentation in high-resolution (HR) remote sensing imagery is one of the most challenging tasks and is more difficult than object detection and semantic segmentation tasks. It aims to predict class labels and pixel-wise instance masks to locate instances in an image. However, there are rare methods currently suitable for instance segmentation in the HR remote sensing images. Meanwhile, it is more difficult to implement instance segmentation due to the complex background of remote sensing images. In this article, a novel instance segmentation approach of HR remote sensing imagery based on Cascade Mask R-CNN is proposed, which is called a high-quality instance segmentation network (HQ-ISNet). In this scheme, the HQ-ISNet exploits a HR feature pyramid network (HRFPN) to fully utilize multi-level feature maps and maintain HR feature maps for remote sensing images' instance segmentation. Next, to refine mask information flow between mask branches, the instance segmentation network version 2 (ISNetV2) is proposed to promote further improvements in mask prediction accuracy. Then, we construct a new, more challenging dataset based on the synthetic aperture radar (SAR) ship detection dataset (SSDD) and the Northwestern Polytechnical University very-high-resolution 10-class geospatial object detection dataset (NWPU VHR-10) for remote sensing images instance segmentation which can be used as a benchmark for evaluating instance segmentation algorithms in the high-resolution remote sensing images. Finally, extensive experimental analyses and comparisons on the SSDD and the NWPU VHR-10 dataset show that (1) the HRFPN makes the predicted instance masks more accurate, which can effectively enhance the instance segmentation performance of the high-resolution remote sensing imagery; (2) the ISNetV2 is effective and promotes further improvements in mask prediction accuracy; (3) our proposed framework HQ-ISNet is effective and more accurate for instance segmentation in the remote sensing imagery than the existing algorithms.

show abstract

Section: Backbone Network and Rpnmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

HQ-ISNet: High-Quality Instance Segmentation for Remote Sensing Imagery

Wei

Liu

et al. 2020

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Object detection can be divided into traditional handcrafted feature-based object detection [9,10] and deep-learning-based object detection [11,12]. It focuses on the target-feature extraction method design for handcrafted feature-based object detection; however, it is still hard to satisfy different conditions, which leads to most of these kinds of methods just being used for limited environment [13][14][15].…”

Section: Introductionmentioning

confidence: 99%

UAV-YOLO: Small Object Detection on Unmanned Aerial Vehicle Perspective

Liu

Wang

Zhou³

et al. 2020

Sensors

238

View full text Add to dashboard Cite

Object detection, as a fundamental task in computer vision, has been developed enormously, but is still challenging work, especially for Unmanned Aerial Vehicle (UAV) perspective due to small scale of the target. In this study, the authors develop a special detection method for small objects in UAV perspective. Based on YOLOv3, the Resblock in darknet is first optimized by concatenating two ResNet units that have the same width and height. Then, the entire darknet structure is improved by increasing convolution operation at an early layer to enrich spatial information. Both these two optimizations can enlarge the receptive filed. Furthermore, UAV-viewed dataset is collected to UAV perspective or small object detection. An optimized training method is also proposed based on collected UAV-viewed dataset. The experimental results on public dataset and our collected UAV-viewed dataset show distinct performance improvement on small object detection with keeping the same level performance on normal dataset, which means our proposed method adapts to different kinds of conditions.

show abstract

“…Compared with low spatial resolution remote sensing images, an HSRRS image contains more details of ground objects and more complex spatial patterns [4][5][6][7][8]. Therefore, sub-meter level HSRRS images have been applied in many areas such as land resources planning [9,10], geospatial object detection [11][12][13], and environmental monitoring [14]. HSRRS-image-based scene classification has attracted increasing attention in the remote sensing community [15][16][17][18][19][20].…”

Section: Introductionmentioning

confidence: 99%

Combing Triple-Part Features of Convolutional Neural Networks for Scene Classification in Remote Sensing

Hong

2019

Remote Sensing

View full text Add to dashboard Cite

High spatial resolution remote sensing (HSRRS) images contain complex geometrical structures and spatial patterns, and thus HSRRS scene classification has become a significant challenge in the remote sensing community. In recent years, convolutional neural network (CNN)-based methods have attracted tremendous attention and obtained excellent performance in scene classification. However, traditional CNN-based methods focus on processing original red-green-blue (RGB) image-based features or CNN-based single-layer features to achieve the scene representation, and ignore that texture images or each layer of CNNs contain discriminating information. To address the above-mentioned drawbacks, a CaffeNet-based method termed CTFCNN is proposed to effectively explore the discriminating ability of a pre-trained CNN in this paper. At first, the pretrained CNN model is employed as a feature extractor to obtain convolutional features from multiple layers, fully connected (FC) features, and local binary pattern (LBP)-based FC features. Then, a new improved bag-of-view-word (iBoVW) coding method is developed to represent the discriminating information from each convolutional layer. Finally, weighted concatenation is employed to combine different features for classification. Experiments on the UC-Merced dataset and Aerial Image Dataset (AID) demonstrate that the proposed CTFCNN method performs significantly better than some state-of-the-art methods, and the overall accuracy can reach 98.44% and 94.91%, respectively. This indicates that the proposed framework can provide a discriminating description for HSRRS images. 2 of 23 color features, spectral features, and multi-feature fusion [23,24]. However, these hand-crafted features are limited in describing complex scenes of HSRRS, which will affect the classification performance. Compared with the low-level methods, the mid-level methods aim to obtain a global representation of a scene by encoding local descriptors, e.g., scale-invariant feature transform, histogram of oriented gradient and color histogram. The bag-of-view-word (BoVW) model is one of the most popular feature encoding approaches [25]. Due to its simplicity and efficiency, the BoVW model is widely applied for mid-level scene description [26][27][28][29]. However, the quantization error of the BoVW method is large, and some important information may be lost. Therefore, many feature coding methods were developed to reduce the reconstruction error, including improved Fisher kernel (IFK) [30], vectors of locally aggregated descriptors (VLAD) [31], spatial pyramid matching kernel (SPM) [32], locality-constrained linear coding (LLC) [33], latent semantic analysis (LSA), probabilistic latent semantic analysis (pLSA) [34,35], and latent dirichlet allocation (LDA) [35]. However, both low-level and mid-level methods are mainly based on hand-crafted features, which are difficult to effectively describe in HSRRS scene images with complex land-cover/land-use (LULC) situations.In recent years, deep-learning-based methods have made a...

show abstract

Multi-scale object detection in remote sensing imagery with convolutional neural networks

Cited by 388 publications

References 51 publications

HQ-ISNet: High-Quality Instance Segmentation for Remote Sensing Imagery

HQ-ISNet: High-Quality Instance Segmentation for Remote Sensing Imagery

UAV-YOLO: Small Object Detection on Unmanned Aerial Vehicle Perspective

Combing Triple-Part Features of Convolutional Neural Networks for Scene Classification in Remote Sensing

Contact Info

Product

Resources

About