Marine Vessel Re-Identification: A Large-Scale Dataset and Global-and-Local Fusion-Based Discriminative Feature Learning

Abstract

A marine vessel re-identification system has to determine whether or not different images represent the same vessel. Accurate vessel re-identification improves onshore closed-circuit television monitoring in a vessel traffic services system as well as onboard surveillance of surrounding vessels. However, because ships are rigid bodies and the marine environment is harsh, the accurate re-identification of vessels at sea can be very difficult. We describe a marine vessel-re-identification framework, Globaland-Local Fusion-based Multi-view Feature Learning (GLF-MVFL), which is based on a combination of global and fine-grained local features. GLF-MVFL combines cross-entropy loss with our newly-developed orientation-guided quintuplet loss. We exploit intrinsic features of marine vessels to optimize multi-view representation learning for re-identification. GLF-MVFL uses ResNet-50 as the backbone network to extract features for simultaneous quintuple input. It detects and discriminates between features and estimates viewpoints to form a comprehensive re-identification framework. We created an annotated large-scale vessel retrieval dataset, VesselID-539, which contains images from viewpoints similar to those of an autonomous surface vessel, to use in evaluating the performance of the model. Extensive experiments and analysis of the results obtained from using VesselID-539 demonstrate that our approach significantly increases the accuracy of vessel re-identification and is more effective and robust for images from different viewpoints than other approaches. INDEX TERMS Autonomous surface vessel (ASV), maritime surveillance, VesselID-539 dataset, multi views, vessel re-identification (V-ReID).

Marine Vision-Based Situational Awareness Using Discriminative Deep Learning: A Survey

“…Target re-identification is an area of image retrieval that mostly focuses on pedestrians and vehicles. Google Scholar searches using keywords such as "Ship + Re-identification", "Ship + Re identification", "Vessel + Re-identification", and "Vessel + Re identification" show that there are few reports on related achievements in the ship field [47][48][49][50], two of which were published by authors of this paper. We define vessel re-identification (also referred to as ship-face recognition) in a manner analogous to the re-identificaton of pedestrians and vehicles and of the target ship images in a given probe set as the process of using computer vision to retrieve the ship in the gallery set of cross frame or cross camera images.…”

“…Such re-identification can be introduced into the multitarget tracking pipeline for long term tracking. Figure 4 shows the four key steps of vessel re-identification [49]: ship detection, feature extraction, feature transformation, and feature similarity metric learning.…”

“…Such re-identification can be introduced into the multitarget tracking pipeline for long term tracking. Figure 4 shows the four key steps of vessel re-identification [49]: ship detection, feature extraction, feature transformation, and feature similarity metric learning. Visual information of ships is different from the visual information of pedestrians, vehicles, and even rare animals in two significant areas when considering re-identification.…”

Maritime vessel re-identification: novel VR-VCA dataset and a multi-branch architecture MVR-net

“…In [11], authors base their work on [14] and extend the method with various multiquery strategies. In [29], authors introduce a new dataset, as well as a novel approach that employs global-and-local fusion-based discriminative feature learning. This method combines CE loss with a novel, orientation-guided quintuplet loss and performs multi-view representation learning for re-ID.…”

Deep Learning for Vessel Detection and Identification From Spaceborne Optical Imagery

“…More modern approaches based on deep learning are so far concerned with vessel (re-)identification from natural images, such as those acquired by in-situ port cameras, devices onboard vessels, etc. (Gundogdu et al, 2017;Qiao et al, 2020). Twin neural networks (Hoffer and Ailon, 2015;Koch, 2015), originally referred to as "Siamese", have been a key step forward in the field of contrastive/metric learning (Chen et al, 2020).…”