Towards Accurate Oriented Object Detection in Aerial Images with Adaptive Multi-level Feature Fusion

Zhen, Peining; Wang, Shuqi; Zhang, Suming; Yan, Xiaotao; Wang, Wei; Ji, Zhigang; Chen, Hai-Bao

doi:10.1145/3513133

Cited by 17 publications

(5 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Oriented R-CNN [26] R50 80.87 R3Det-GWD [44] R152 80.19 R3Det-KLD [45] R152 80.63 AFF-Det [41] R50 80.73 KFIoU [46] Swin-T 80.93 RVSA [47] ViT-B 81.01 S 2 ANet [1] R50 79.42 ReDet [38] Re-R50 80.10 AOPG [39] R50 80.66 R3Det [2] R152 76.47 G-Rep [32] Swin-T 80.16…”

Section: Methods Backbone Mapmentioning

confidence: 99%

HVConv: Horizontal and Vertical Convolution for Remote Sensing Object Detection

Chen,

Lin,

Huang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

Generally, the interesting objects in aerial images are completely different from objects in nature, and the remote sensing objects in particular tend to be more distinctive in aspect ratio. The existing convolutional networks have equal aspect ratios of the receptive fields, which leads to receptive fields either containing non-relevant information or being unable to fully cover the entire object. To this end, we propose Horizontal and Vertical Convolution, which is a plug-and-play module to address different aspect ratio problems. In our method, we introduce horizontal convolution and vertical convolution to expand the receptive fields in the horizontal and vertical directions, respectively, to reduce redundant receptive fields, so that remote sensing objects with different aspect ratios can achieve better receptive fields coverage, thereby achieving more accurate feature representation. In addition, we design an attention module to dynamically aggregate these two sub-modules to achieve more accurate feature coverage. Extensive experimental results on the DOTA and HRSC2016 datasets show that our HVConv achieves accuracy improvements in diverse detection architectures and obtains SOTA accuracy (mAP score of 77.60% with DOTA single-scale training and mAP score of 81.07% with DOTA multi-scale training). Various ablation studies were conducted as well, which is enough to verify the effectiveness of our model.

show abstract

Section: Methods Backbone Mapmentioning

confidence: 99%

HVConv: Horizontal and Vertical Convolution for Remote Sensing Object Detection

Chen,

Lin,

Huang

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…CGNet [ 39 ] uses self-attention to enhance communication between pyramid levels. AFF-Det [ 14 , 18 ] maps ROIs to all levels and applies a unified supervisory signal to alleviate the semantic gap. The authors of [ 16 , 17 ] perform feature refinement after scaling.…”

Section: Methodsmentioning

confidence: 99%

“…The mainstream method [ 9 ] conveys the semantic information layer-by-layer using lateral connections. Other improved works [ 13 , 14 , 15 , 16 , 17 , 18 ] apply semantic reconfiguration to single-scale aggregated features resampled from all pyramid levels. However, we find that these methods have certain shortcomings: First, fusing all pyramid features with different receptive fields into a single scale introduces a lot of irrelevant semantic information.…”

Section: Methodsmentioning

confidence: 99%

“…However, these methods rely on an additional refine head, which greatly impacts inference efficiency and parameters. Another direction is the fusion method based on feature resampling: [ 13 , 14 , 15 , 16 , 17 , 18 ] aggregate features from all pyramid levels and finally predict at the specified feature scale. The disadvantage of these methods is that resampling features to a single scale weakens the scale sensitivity of the network.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Shared-Weight-Based Multi-Dimensional Feature Alignment Network for Oriented Object Detection in Remote Sensing Imagery

Zhu

2022

Sensors

View full text Add to dashboard Cite

Arbitrarily Oriented Object Detection in aerial images is a highly challenging task in computer vision. The mainstream methods are based on the feature pyramid, while for remote-sensing targets, the misalignment of multi-scale features is always a thorny problem. In this article, we address the feature misalignment problem of oriented object detection from three dimensions: spatial, axial, and semantic. First, for the spatial misalignment problem, we design an intra-level alignment network based on leading features that can synchronize the location information of different pyramid features by sparse sampling. For multi-oriented aerial targets, we propose an axially aware convolution to solve the mismatch between the traditional sampling method and the orientation of instances. With the proposed collaborative optimization strategy based on shared weights, the above two modules can achieve coarse-to-fine feature alignment in spatial and axial dimensions. Last but not least, we propose a hierarchical-wise semantic alignment network to address the semantic gap between pyramid features that can cope with remote-sensing targets at varying scales by endowing the feature map with global semantic perception across pyramid levels. Extensive experiments on several challenging aerial benchmarks show state-of-the-art accuracy and appreciable inference speed. Specifically, we achieve a mean Average Precision (mAP) of 78.11% on DOTA, 90.10% on HRSC2016, and 90.29% on UCAS-AOD.

show abstract

“…Different from natural images, UAVs usually capture aerial images under varying illumination and uncontrolled outdoor conditions, which requires object detection models with strong robustness 20 , 21 . Existing methods are mainly carried out in terms of model structure and labeled data 22 .…”

Section: Related Workmentioning

confidence: 99%

Expert teacher based on foundation image segmentation model for object detection in aerial images

Yu,

Sun,

Cheng

2023

Sci Rep

View full text Add to dashboard Cite

Despite the remarkable progress of general object detection, the lack of labeled aerial images limits the robustness and generalization of the detector. Teacher–student learning is a feasible solution on natural image domain, but few works focus on unlabeled aerial images. Inspired by foundation models with the powerful generalization in computer vision field, we propose an expert teacher framework based on foundation image segmentation model called ET-FSM. Our approach provides the performance gains for the student detector by generating high-quality pseudo-labels for unlabeled aerial images. In the ET-FSM, we design the binary detector with expert guidance mechanism to sufficiently leverage the extra knowledge obtained from the foundation image segmentation model, which accurately detects object positions in the complex backgrounds. Also, we present the momentum contrast classification module to distinguish confused object categories in aerial images. To demonstrate the effectiveness of the proposed method, we construct an unlabeled aerial image dataset covering various scenes. The experiments are conducted on diverse types of student detectors. The results show that the proposed approach achieves superior performance compared to existing methods, and allows the student detector to achieve fully supervised performance with much less labeled aerial images. Our dataset and code are available at https://github.com/cq100/ET-FSM.

show abstract

Towards Accurate Oriented Object Detection in Aerial Images with Adaptive Multi-level Feature Fusion

Cited by 17 publications

References 36 publications

HVConv: Horizontal and Vertical Convolution for Remote Sensing Object Detection

HVConv: Horizontal and Vertical Convolution for Remote Sensing Object Detection

Shared-Weight-Based Multi-Dimensional Feature Alignment Network for Oriented Object Detection in Remote Sensing Imagery

Expert teacher based on foundation image segmentation model for object detection in aerial images

Contact Info

Product

Resources

About