Multiscale Semantic Fusion-Guided Fractal Convolutional Object Detection Network for Optical Remote Sensing Imagery

Zhang, Tong; Zhuang, Yin; Wang, Guanqun; Dong, Shan; Chen, He; Li, Lianlin

doi:10.1109/tgrs.2021.3108476

Cited by 47 publications

(9 citation statements)

References 96 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This module helps capture information at different scales and spatial contexts, enabling a more comprehensive understanding of the targets and improving the detection accuracy for objects of varying sizes in remote sensing images. MSFC-Net [92] is a single-level detection method using the Composite Semantic Feature Fusion (CSFF) method for generating valid semantic descriptions. It utilizes a Fractal Convolutional (FC) regression layer for adaptive regression of irregular aspect ratio multi-scale bounding boxes.…”

Section: Methodsmentioning

confidence: 99%

A Survey of Small Object Detection Based on Deep Learning in Aerial Images

Wang

Chen

2023

Preprint

View full text Add to dashboard Cite

Small object detection poses a formidable challenge in the field of computer vision, particularly when it comes to analyzing aerial remote sensing images. Despite the rapid development of deep learning and significant progress in detection techniques in natural scenes, the migration of these algorithms to aerial images has not met expectations. This is primarily due to limitations in imaging acquisition conditions, including small target size, viewpoint specificity, background complexity, as well as scale and orientation diversity. Although the increasing application of deep learning-based algorithms to overcome these problems, few studies have summarized the optimization of different deep learning strategies used for small target detection in aerial images. Therefore, this paper aims to explore the application of deep learning methods for small object detection in aerial images. The primary challenges in small object detection in aerial images will be summarized. Next, a meticulous analysis and categorization of the prevailing deep learning optimization strategies employed to surmount the challenges encountered in aerial image detection is undertaken. Following that, we provide a comprehensive presentation of the object detection datasets utilized in aerial remote sensing images, along with the evaluation metrics employed. Additionally, we furnish experimental data pertaining to the currently proposed detection algorithms. Finally, the advantages and disadvantages of various optimization strategies and potential development trends are discussed. Hopefully, it can provide a reference for researchers in this field.

show abstract

Section: Methodsmentioning

confidence: 99%

A Survey of Small Object Detection Based on Deep Learning in Aerial Images

Wang

Chen

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Wang et al [41] proposed an absorption pruning method for the object detection network in remote-sensing images. Tong et al [42] designed a compound semantic feature fusion method to generate an effective semantic description for better pixel-wise object center point interpretation. Ma et al [43] proposed a feature split-merge-enhancement network (SME-Net) to handle objects with significant scale differences.…”

Section: Label Assignment Strategiesmentioning

confidence: 99%

G-Rep: Gaussian Representation for Arbitrary-Oriented Object Detection

et al. 2023

View full text Add to dashboard Cite

Typical representations for arbitrary-oriented object detection tasks include the oriented bounding box (OBB), the quadrilateral bounding box (QBB), and the point set (PointSet). Each representation encounters problems that correspond to its characteristics, such as boundary discontinuity, square-like problems, representation ambiguity, and isolated points, which lead to inaccurate detection. Although many effective strategies have been proposed for various representations, there is still no unified solution. Current detection methods based on Gaussian modeling have demonstrated the possibility of resolving this dilemma; however, they remain limited to OBB. To go further, in this paper, we propose a unified Gaussian representation called G-Rep to construct Gaussian distributions for OBB, QBB, and PointSet, which achieves a unified solution to various representations and problems. Specifically, PointSet- or QBB-based object representations are converted into Gaussian distributions and their parameters are optimized using the maximum likelihood estimation algorithm. Then, three optional Gaussian metrics are explored to optimize the regression loss of the detector because of their excellent parameter optimization mechanisms. Furthermore, we also use Gaussian metrics for sampling to align label assignment and regression loss. Experimental results obtained on several publicly available datasets, such as DOTA, HRSC2016, UCAS-AOD, and ICDAR2015, show the excellent performance of the proposed method for arbitrary-oriented object detection.

show abstract

“…Among these methods, some classical detectors, such as Faster-RCNN [88], YOLOv5 [89], Mask-RCNN [71], PANet [90] and CenterNet [91] are selected. Meanwhile, we also compare with some SOTA detectors from RSD including MSFC-Net [92], CANet [93] and FSoD [94]. As reported in the 9th and 14th rows of Table 6, compared with the original version of Mask-RCNN [71], our method achieves 3% mAP improvement without bells and whistles based on ViT-B [14].…”

Section: Object Detectionmentioning

confidence: 99%

Consecutive Pre-Training: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domain

et al. 2022

View full text Add to dashboard Cite

Currently, under supervised learning, a model pre-trained by a large-scale nature scene dataset and then fine-tuned on a few specific task labeling data is the paradigm that has dominated knowledge transfer learning. Unfortunately, due to different categories of imaging data and stiff challenges of data annotation, there is not a large enough and uniform remote sensing dataset to support large-scale pre-training in the remote sensing domain (RSD). Moreover, pre-training models on large-scale nature scene datasets by supervised learning and then directly fine-tuning on diverse downstream tasks seems to be a crude method, which is easily affected by inevitable incorrect labeling, severe domain gaps and task-aware discrepancies. Thus, in this paper, considering the self-supervised pre-training and powerful vision transformer (ViT) architecture, a concise and effective knowledge transfer learning strategy called ConSecutive Pre-Training (CSPT) is proposed based on the idea of not stopping pre-training in natural language processing (NLP), which can gradually bridge the domain gap and transfer large-scale data knowledge to any specific domain (e.g., from nature scene domain to RSD) In addition, the proposed CSPT also can release the huge potential of unlabeled data for task-aware model training. Finally, extensive experiments were carried out on twelve remote sensing datasets involving three types of downstream tasks (e.g., scene classification, object detection and land cover classification) and two types of imaging data (e.g., optical and synthetic aperture radar (SAR)). The results show that by utilizing the proposed CSPT for task-aware model training, almost all downstream tasks in the RSD can outperform the previous knowledge transfer learning strategies based on model pre-training without any expensive manually labeling and even surpass the state-of-the-art (SOTA) performance without any careful network architecture designing.

show abstract

Multiscale Semantic Fusion-Guided Fractal Convolutional Object Detection Network for Optical Remote Sensing Imagery

Cited by 47 publications

References 96 publications

A Survey of Small Object Detection Based on Deep Learning in Aerial Images

A Survey of Small Object Detection Based on Deep Learning in Aerial Images

G-Rep: Gaussian Representation for Arbitrary-Oriented Object Detection

Consecutive Pre-Training: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domain

Contact Info

Product

Resources

About