Rotationally Unconstrained Region Proposals for Ship Target Segmentation in Optical Remote Sensing

Abstract

The segmentation of ship targets in remote sensing images is of great importance in both military and civil fields. The existing methods often suffer from a low overlap between region proposals, which are constrained to be the horizontal rectangles and the ground truth. This enhances the problems of background noise and results in more false positives and a lower recall. Hence, we propose a method to generate the rotated bounding boxes that are more appropriate for ship segmentation. The proposed method is based on the Mask R-CNN framework, and the key contribution lies in the approach to predefine the rotated bounding boxes and generate the rotationally unconstrained region proposals. Specifically, for a double-stage proposal generation, a multiangle region proposal layer is designed; following this, an adapted alignment approach is proposed to extract the features for each proposal. In addition to the above-mentioned two proposed methods, we adopt a second stage to refine the proposals with rotation regression and mask prediction. We evaluated the proposed method using a remote sensing dataset with extensively labeled ship targets, and the experimental results show that the proposed method performs better than its competitors. The proposed method significantly increases the recall of densely arranged ships while substantially reducing the number of false positives.INDEX TERMS Optical remote sensing images, instance segmentation, joint learning, ship target segmentation, unconstrained bounding boxes.

Object Detection and Image Segmentation with Deep Learning on Earth Observation Data: A Review—Part II: Applications

“…Images with a very high resolution were especially used to detect ships in situations where they are lying very close together, like in harbors or near harbor berths. In order to separate multiple ships or disentangle ships and harbor structures, rotated bounding boxes were found to be very useful [79,[93][94][95][96][97][98][99][100][101][102][103][104][105]. In contrast to horizontal bounding boxes, they allow a higher signal to noise ratio and hence, better detection performance [95].…”

“…In order to separate multiple ships or disentangle ships and harbor structures, rotated bounding boxes were found to be very useful [79,[93][94][95][96][97][98][99][100][101][102][103][104][105]. In contrast to horizontal bounding boxes, they allow a higher signal to noise ratio and hence, better detection performance [95]. Also, the overlap of nearby bounding boxes decreases, which prevents falsely suppressed bounding boxes of dense conglomerations of ships.…”

“…By combining object detection with further classification or segmentation modules, studies were also able to determine a specific ship class [57,103,107,108], extract instance masks for ships [95,106] or even predict their direction [103,109]. Since ships are relatively small targets against a large non-target background, the sea, weighted cost functions were used to improve training and performance [110] significantly.…”

“…The feature pyramid network (FPN) [365] module is commonly known to enrich extracted features and pass them to the detector on multiple scales. FPN and resembling techniques were leveraged in several studies, which also proposed further modifications of this particular structure [65,80,95,100,124,125,225,227,232,235,236,325,[366][367][368][369][370][371][372][373][374][375]. Other positive effects on detecting objects in remotely sensed images were found in modifications which compensate for the spatial information loss that happens during feature extraction [228,376].…”

Small Ship Detection Based on Hybrid Anchor Structure and Feature Super-Resolution

“…Research on ship target detection technology is of great significance. Due to the great success of object detection algorithms based on convolutional neural networks [1][2][3][4][5][6][7][8][9][10], many researchers use similar techniques to achieve ship detection [11][12][13][14][15][16][17][18][19][20][21]. However, it is still a very challenging task to accurately locate small ships from optical remote sensing images.…”