Siamese Network Based Pelage Pattern Matching for Ringed Seal Re-identification

Abstract

In this paper we propose a method to match pelage patterns of the Saimaa ringed seals enabling the reidentification of individuals. First, the pelage pattern is extracted from the seal's fur using a method based on the Sato tubeness filter. After this, the similarities of the pelage pattern patches are computed using a siamese network trained with a triplet loss function and a large dataset of manually selected patches. The similarities are then used to find the best matching patches from the images in the database of known individuals. Furthermore, we employ the proposed pattern matching method to build a full framework for the ringed seal re-identification, consisting of CNN-based animal segmentation, patch correspondence detection, and ranking the images in the database of known seal individuals based on the similarity to the query image. Our experiments on challenging datasets of Saimaa ringed seals show that the proposed method achieves promising identification results, providing a useful tool for the Saimaa ringed seal monitoring.

Perspectives on Individual Animal Identification from Biology and Computer Vision

“…The authors also explore the impact of increasing the number of individuals and the number of images per individual, both leading to score increases. Nepovinnykh et al (2020) applied metric learning to re-identify Saimaa ringed seals. After segmentation with DeepLab ( Chen et al 2018 ) and subsequent cropping, the authors extracted pelage pattern features with a Sato tubeness filter used as input to their network.…”

“…Indeed, Kshitij and Sai (2020) also showed that—for some species—priming ConvNets with handcrafted features produced better results than using the raw images. Instead of using k-NNs, Nepovinnykh et al (2020) adopt topologically aware heatmaps to identify individual seals—both the query image and the database images are split into patches whose similarity is computed, and among the most similar, topological similarity is checked through angle difference ranking. For 2000 images of 46 seals, the authors achieved a top-1 score of 67.8% and a top-5 score of 88.6%.…”

Comparing Class-Aware and Pairwise Loss Functions for Deep Metric Learning in Wildlife Re-Identification

“…Nepovinnykh et al [ 42 ] adapted a Deeplab [ 20 ] model that was trained on triplet loss to re-identify individuals in a ringed seals population. Deeplab was pre-trained on Pascal VOC dataset [ 43 ], no pair sampling technique was discussed in the Deeplab system.…”

“…The triplet loss approach, coupled with semi-hard pair sampling, resulted in state-of-the-art performance in human face re-identification in experiments [ 29 ]. As reflected in Section 2.6 , triplet loss and contrastive loss have been applied in animal re-identification tasks by Nepovinnykh et al [ 42 ] and van Zyl et al [ 8 ]. The pairwise loss functions demand that meaningful pairs are found during the training phase.…”

Contrastively Learning Visual Attention as Affordance Cues from Demonstrations for Robotic Grasping

“…Siamese neural network can be combined with a ConvNets and trained with a Binary Cross Entropy loss as in [13], or triplet loss as in [26]). Recently deep learning based Siamese neural networks are also applied in many new applications, like face recognition ( [26], [27]), object discovery ( [28], [11]), object co-segmentation ([3], [17], [21]), and re-identification as in [35], [22].…”