A data augmentation method for fine-grained ship detection in remote sensing images

Abstract

The advancement of deep learning has given significant improvement in fine-grained ship detection. However, a major challenge in deep learning based fine-grained ship detection in remote sensing images is the long-tail distribution, not only in the aspect of the number of objects but also in the average instance area of ships. Thus, models tend to be biased towards the categories with more samples or a larger average instance area, i.e., majority classes, and results in low accuracy in recognizing the categories with fewer samples or smaller average instance area, i.e., minority classes. To address this challenge, a novel data augmentation method is proposed in this paper, namely SegRotatePaste. Firstly, a weighted combination-based strategy is proposed by employing the number ratio and the average instance area ratio of ships as evaluation metrics, to identify the minority class and determine the required number of augmentations. Then sea-land segmentation and ship annotations are introduced as auxiliary semantic information to guide the embedding location of ship clips. In addition, various image embedding techniques are leveraged to achieve data augmentation for the minority class. The proposed data augmentation method is evaluated on the HRSC2016 and FGSD2021 datasets. Our method significantly improves the 𝒎𝑨𝑷 of the minority class by 9.3% and 11.5% over the baseline on the HRSC2016 and FGSD2021 datasets, respectively.