ADT-Det: Adaptive Dynamic Refined Single-Stage Transformer Detector for Arbitrary-Oriented Object Detection in Satellite Optical Imagery

Abstract

The detection of arbitrary-oriented and multi-scale objects in satellite optical imagery is an important task in remote sensing and computer vision. Despite significant research efforts, such detection remains largely unsolved due to the diversity of patterns in orientation, scale, aspect ratio, and visual appearance; the dense distribution of objects; and extreme imbalances in categories. In this paper, we propose an adaptive dynamic refined single-stage transformer detector to address the aforementioned challenges, aiming to achieve high recall and speed. Our detector realizes rotated object detection with RetinaNet as the baseline. Firstly, we propose a feature pyramid transformer (FPT) to enhance feature extraction of the rotated object detection framework through a feature interaction mechanism. This is beneficial for the detection of objects with diverse patterns in terms of scale, aspect ratio, visual appearance, and dense distributions. Secondly, we design two special post-processing steps for rotated objects with arbitrary orientations, large aspect ratios and dense distributions. The output features of FPT are fed into post-processing steps. In the first step, it performs the preliminary regression of locations and angle anchors for the refinement step. In the refinement step, it performs adaptive feature refinement first and then gives the final object detection result precisely. The main architecture of the refinement step is dynamic feature refinement (DFR), which is proposed to adaptively adjust the feature map and reconstruct a new feature map for arbitrary-oriented object detection to alleviate the mismatches between rotated bounding boxes and axis-aligned receptive fields. Thirdly, the focus loss is adopted to deal with the category imbalance problem. Experiments on two challenging satellite optical imagery public datasets, DOTA and HRSC2016, demonstrate that the proposed ADT-Det detector achieves a state-of-the-art detection accuracy (79.95% mAP for DOTA and 93.47% mAP for HRSC2016) while running very fast (14.6 fps with a 600 × 600 input image size).

A Survey of Small Object Detection Based on Deep Learning in Aerial Images

“…Problem solved Optimization Strategies single-stage detector [41] Boundary-arbitrary discontinuity FAB+DRBs+CRB DA-Net [42] Boundary-arbitrary discontinuity RFS+RFA MGAR [43] Vague angle representation CAC+FAR+IFL OrtDet [44] Angular periodicity Mean rotational accuracy (mRP) AProNet [45] Angular periodicity Axis-based Angle Learning Arbitrary orientation regression [46] Arbitrary angle Adaptive target orientation regression CFC-Net [47] Arbitrary angle Rotation anchor refinement module Angle encoding mechanism [48] Arbitrary angle Aspect ratio-based bidirectional coding label RH-RCNN [49] Arbitrary angle Distinguish tilted targets AOPDet [50] Rotation object representation Non-sequential angular representation ACE [51] Rotation object representation Directed quadrilateral box Faster R-CNN-based [52] Rotated Region Proposal Majority voting strategy RiDOP [53] Rotated Region Proposal Sliding only two vertices R-RCNN [54] Rotated Region Proposal Directional RoI pooling operation Point RCNN [55] Rotated Region Proposal PointRPN module generates RRoI New anchor-free detector [56] Arbitrary angle Center Boundary Dual-Attention (CBDA) AOPG [57] Arbitrary angle Generates orientation boxes in an anchor-free manner AOPG+FRIoU [58] Arbitrary angle Focal Rotated Intersection over Union(FRIoU) R2YOLOX [59] Arbitrary angle Refined Rotation Module (RRM) DARDet [60] Arbitrary angle ACM+PIoU ADT-Det [61] Inadequate expression of features Feature Pyramid Transformer (FPT) AFA-FPN [62] Inadequate expression of features Employs RROI to rotate the horizontal frames RINet [63] Inadequate expression of features Flexible multi-branch online detector improvement FoRDet [64] Inadequate expression of features Foreground Relationship module (FRL)…”

“…The effectiveness of detecting rotating targets is improved by using a multi-branch network structure, which allows for multi-level flexible perception of images. The Feature Pyramid Transformer [61] (FPT) mechanism enhances feature extraction in rotating target detection through feature interaction. The rotation branch in the Attention-based Feature Alignment FPN [62] (AFA-FPN) method employs RROI to rotate the horizontal frames obtained by RPN, thereby preventing missed detections of small targets with dense distribution and arbitrary orientation.…”

Table Tennis Track Detection Based on Temporal Feature Multiplexing Network

“…Firstly, the feature map with parameter size was equally divided into a sequence of feature blocks , where p was the length and width of the feature blocks and N was the number of feature blocks partitioned from the feature map. Then, each feature block was mapped to D dimensions by a linear transformation to obtain , with each one-dimensional feature block represented as a token [ 52 , 53 , 54 ]. Afterwards, the tokens were fed into the Transformer model; the internal computational structure of the Transformer was a multi-layered, parallel computation, primarily comprising of two sections, the first being a multi-headed self-attention layer and the second being a multi-layer perceptron [ 55 , 56 ], as shown in Figure 2 .…”

Oriented Object Detection in Remote Sensing Images with Anchor-Free Oriented Region Proposal Network

“…Based on RetinaNet [19], ADT-Det [36] uses a feature pyramid transformer that enhances features through feature interaction with multiple scales and layers. S 2 A-Net [29] utilizes a feature alignment module for full feature alignment and an oriented detection module to alleviate the inconsistency between classification and regression.…”