A2RMNet: Adaptively Aspect Ratio Multi-Scale Network for Object Detection in Remote Sensing Images

Qiu, Heqian; Li, Hongliang; Wu, Qingbo; Meng, Fanman; Ngan, King Ngi; Shi, Hao

doi:10.3390/rs11131594

Cited by 72 publications

(36 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of proposal-region-based methods [2,22,[41][42][43]47, 50] are based on the "listener" strategy, which first combines language features with visual features of proposal regions, and then select the target region that best matches the input expression from these proposals. The proposal regions are typically extracted by a pretrained object detector (e.g., Faster R-CNN [34], Mask R-CNN [9] and others [4,17,[30][31][32]). To align visual regions with the expression more powerful, methods in [22,47] proposed to decompose the expression into three components (subject, localization and relationship), and leveraged cross-modality attentions to focus on relevant regions.…”

Section: Related Workmentioning

confidence: 99%

Language-Aware Fine-Grained Object Representation for Referring Expression Comprehension

Qiu

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

Referring expression comprehension expects to accurately locate an object described by a language expression, which requires precise language-aware visual object representations. However, existing methods usually use rectangular object representations, such as object proposal regions and grid regions. They ignore some finegrained object information like shapes and poses, which are often described in language expressions and important to localize objects. Additionally, rectangular object regions usually contain background contents and irrelevant foreground features, which also decrease the localization performance. To address these problems, we propose a language-aware deformable convolution model (LDC) to learn language-aware fine-grained object representations. Rather than extracting rectangular object representations, LDC adaptively samples a set of key points based on the image and language to represent objects. This type of object representations can capture more fine-grained object information (e.g., shapes and poses) and suppress noises in accordance with language and thus, boosts the object localization performance. Based on the language-aware finegrained object representation, we next design a bidirectional interaction model (BIM) that leverages a modified co-attention mechanism to build cross-modal bidirectional interactions to further improve the language and object representations. Furthermore, we propose a hierarchical fine-grained representation network (HFRN) to learn language-aware fine-grained object representations and cross-modal bidirectional interactions at local word level and global sentence level, respectively. Our proposed method outperforms the state-of-the-art methods on the RefCOCO, RefCOCO+ and Ref-COCOg datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

Language-Aware Fine-Grained Object Representation for Referring Expression Comprehension

Qiu

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

“…These parameters are usually selected by trial-and-error, which requires expert knowledge and is a time-consuming and uncertain task [12,55]. Although some automatic techniques, like the Taguchi optimization method [55], have been introduced for defining the optimal parameters for the segmentation process, the process of detecting optimal objects is still a challenging task, mostly because of the diversity in the sizes and shapes of the target features [56].…”

Section: Multi-scale Image Segmentationmentioning

confidence: 99%

Landslide Detection Using Multi-Scale Image Segmentation and Different Machine Learning Models in the Higher Himalayas

et al. 2019

View full text Add to dashboard Cite

Landslides represent a severe hazard in many areas of the world. Accurate landslide maps are needed to document the occurrence and extent of landslides and to investigate their distribution, types, and the pattern of slope failures. Landslide maps are also crucial for determining landslide susceptibility and risk. Satellite data have been widely used for such investigations—next to data from airborne or unmanned aerial vehicle (UAV)-borne campaigns and Digital Elevation Models (DEMs). We have developed a methodology that incorporates object-based image analysis (OBIA) with three machine learning (ML) methods, namely, the multilayer perceptron neural network (MLP-NN) and random forest (RF), for landslide detection. We identified the optimal scale parameters (SP) and used them for multi-scale segmentation and further analysis. We evaluated the resulting objects using the object pureness index (OPI), object matching index (OMI), and object fitness index (OFI) measures. We then applied two different methods to optimize the landslide detection task: (a) an ensemble method of stacking that combines the different ML methods for improving the performance, and (b) Dempster–Shafer theory (DST), to combine the multi-scale segmentation and classification results. Through the combination of three ML methods and the multi-scale approach, the framework enhanced landslide detection when it was tested for detecting earthquake-triggered landslides in Rasuwa district, Nepal. PlanetScope optical satellite images and a DEM were used, along with the derived landslide conditioning factors. Different accuracy assessment measures were used to compare the results against a field-based landslide inventory. All ML methods yielded the highest overall accuracies ranging from 83.3% to 87.2% when using objects with the optimal SP compared to other SPs. However, applying DST to combine the multi-scale results of each ML method significantly increased the overall accuracies to almost 90%. Overall, the integration of OBIA with ML methods resulted in appropriate landslide detections, but using the optimal SP and ML method is crucial for success.

show abstract

“…With the rapid development of deep convolutional neural networks (CNNs) [1] in recent years, the conventional object detection methods [2,3] have made some remarkable achievements in natural images. However, due to the huge scale variations of the vast majority of objects and the compact distribution of many small objects in remote sensing images, it still remains a tremendous challenge for locating and predicting the target objects [4,5].…”

Section: Introductionmentioning

confidence: 99%

“…However, the resizing process may result in small objects becoming smaller and more likely to be lost in the deeper layers. For solving this problem, the general solution is to simply cut large-scale images into small chunks [5,16]. However, when the cut images include relatively large objects, such as ground track field, these objects may be broken up into small pieces and make the network hard recognize.…”

Section: Introductionmentioning

confidence: 99%

Extended Feature Pyramid Network with Adaptive Scale Training Strategy and Anchors for Object Detection in Aerial Images

Guo¹,

Li²,

Gong³

et al. 2020

Remote Sensing

View full text Add to dashboard Cite

Multi-scale object detection is a basic challenge in computer vision. Although many advanced methods based on convolutional neural networks have succeeded in natural images, the progress in aerial images has been relatively slow mainly due to the considerably huge scale variations of objects and many densely distributed small objects. In this paper, considering that the semantic information of the small objects may be weakened or even disappear in the deeper layers of neural network, we propose a new detection framework called Extended Feature Pyramid Network (EFPN) for strengthening the information extraction ability of the neural network. In the EFPN, we first design the multi-branched dilated bottleneck (MBDB) module in the lateral connections to capture much more semantic information. Then, we further devise an attention pathway for better locating the objects. Finally, an augmented bottom-up pathway is conducted for making shallow layer information easier to spread and further improving performance. Moreover, we present an adaptive scale training strategy to enable the network to better recognize multi-scale objects. Meanwhile, we present a novel clustering method to achieve adaptive anchors and make the neural network better learn data features. Experiments on the public aerial datasets indicate that the presented method obtain state-of-the-art performance.Remote Sens. 2020, 12, 784 2 of 22 high-level semantic information at each scale. This structure displays an obvious improvement as a common feature extractor in some practical applications. However, since large-scale objects are usually produced and predicted in the deeper convolution layers of the FPN, the boundaries of these objects might be too fuzzy to obtain accurate regression. Furthermore, the FPN usually predicts small-scale objects in the shallower layers with low semantic information which might not be enough to identify the class of the objects. The designer of the FPN has been aware of this problem and adopted a top-down structure with lateral connections to fuse shallow layers and high-level semantic information to relieve it. However, if the small-scale objects disappear in the deep convolution layers, the context information cues will disappear at the same time.

show abstract

A2RMNet: Adaptively Aspect Ratio Multi-Scale Network for Object Detection in Remote Sensing Images

Cited by 72 publications

References 43 publications

Language-Aware Fine-Grained Object Representation for Referring Expression Comprehension

Language-Aware Fine-Grained Object Representation for Referring Expression Comprehension

Landslide Detection Using Multi-Scale Image Segmentation and Different Machine Learning Models in the Higher Himalayas

Extended Feature Pyramid Network with Adaptive Scale Training Strategy and Anchors for Object Detection in Aerial Images

Contact Info

Product

Resources

About