Method for Whale Re-identification Based on Siamese Nets and Adversarial Training

Wang, W.; Solovyev, Roman; Stempkovsky, A. L.; Telpukhov, Dmitry; Volkov, A.

doi:10.3103/s1060992x20020058

Cited by 4 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to address these problems, most studies either use traditional classifiers 42 , 48 (e.g. SVM), which do not require such large data volumes compared to deep learning methods, but usually also provide worse classification results, or apply Deep Metric Learning 38 , 40 , 45 , 46 , 49 , 51 , 52 , 61 , 64 , especially in combination with the triplet loss 71 – 73 . Considering the aforementioned difficulties regarding the initial usage of the triplet loss and identification of appropriate triplets, traditional supervised classification was performed as an initial step.…”

Section: Discussionmentioning

confidence: 99%

“…detection), meet the standards for robust individual classification. A number of studies performed Deep Metric Learning along with the triplet loss 71 – 73 , modifications of it, and/or combinations with other loss functions 38 , 40 , 45 , 46 , 49 , 51 , 52 , 64 . However, specification of appropriate hard and semi-hard triplets 73 is extremely challenging, since: (1) killer whale individuals have been recorded from both body sides, resulting in different animal orientations besides potential deviating natural markings 45 , (2) natural identifiers change over time (growth, acquisition of scars, etc.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FIN-PRINT a fully-automated multi-stage deep-learning-based framework for the individual recognition of killer whales

Bergler

Gebhard

Towers

et al. 2021

Sci Rep

View full text Add to dashboard Cite

Biometric identification techniques such as photo-identification require an array of unique natural markings to identify individuals. From 1975 to present, Bigg’s killer whales have been photo-identified along the west coast of North America, resulting in one of the largest and longest-running cetacean photo-identification datasets. However, data maintenance and analysis are extremely time and resource consuming. This study transfers the procedure of killer whale image identification into a fully automated, multi-stage, deep learning framework, entitled FIN-PRINT. It is composed of multiple sequentially ordered sub-components. FIN-PRINT is trained and evaluated on a dataset collected over an 8-year period (2011–2018) in the coastal waters off western North America, including 121,000 human-annotated identification images of Bigg’s killer whales. At first, object detection is performed to identify unique killer whale markings, resulting in 94.4% recall, 94.1% precision, and 93.4% mean-average-precision (mAP). Second, all previously identified natural killer whale markings are extracted. The third step introduces a data enhancement mechanism by filtering between valid and invalid markings from previous processing levels, achieving 92.8% recall, 97.5%, precision, and 95.2% accuracy. The fourth and final step involves multi-class individual recognition. When evaluated on the network test set, it achieved an accuracy of 92.5% with 97.2% top-3 unweighted accuracy (TUA) for the 100 most commonly photo-identified killer whales. Additionally, the method achieved an accuracy of 84.5% and a TUA of 92.9% when applied to the entire 2018 image collection of the 100 most common killer whales. The source code of FIN-PRINT can be adapted to other species and will be publicly available.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

FIN-PRINT a fully-automated multi-stage deep-learning-based framework for the individual recognition of killer whales

Bergler

Gebhard

Towers

et al. 2021

Sci Rep

View full text Add to dashboard Cite

show abstract

“…They trained and used a YOLOv3 model [42] for detecting dorsal fins and a multi-class classification ResNet-34 model [43] for determining the identity of the killer whales. Wang et al proposed to use a Siamese network and adversarial training to identify whales by their flukes [44] and Nepovinnykh et al trained a Siamese network for Saimaa Ringed Seal re-identification [45]. In another work concerning seals, Chelak et al [20] proposed a new global pooling technique named EDEN and illustrated that the deep features of a modified and fine-tuned ResNet-18 model are suitable for re-identifying Saimaa Ringed Seals.…”

Section: Related Workmentioning

confidence: 99%

Finding Nemo’s Giant Cousin: Keypoint Matching for Robust Re-Identification of Giant Sunfish

Pedersen

Nyegaard²,

Moeslund

2023

JMSE

View full text Add to dashboard Cite

The Giant Sunfish (Mola alexandrini) has unique patterns on its body, which allow for individual identification. By continuously gathering and matching images, it is possible to monitor and track individuals across location and time. However, matching images manually is a tedious and time-consuming task. To automate the process, we propose a pipeline based on finding and matching keypoints between image pairs. We evaluate our pipeline with four different keypoint descriptors, namely ORB, SIFT, RootSIFT, and SuperPoint, and demonstrate that the number of matching keypoints between a pair of images is a strong indicator for the likelihood that they contain the same individual. The best results are obtained with RootSIFT, which achieves an mAP of 75.91% on our test dataset (TinyMola+) without training or fine-tuning any parts of the pipeline. Furthermore, we show that the pipeline generalizes to other domains, such as re-identification of seals and cows. Lastly, we discuss the impracticality of a ranking-based output for real-life tasks and propose an alternative approach by viewing re-identification as a binary classification. We show that the pipeline can be easily modified with minimal fine-tuning to provide a binary output with a precision of 98% and recall of 44% on the TinyMola+ dataset, which basically eliminates the need for time-consuming manual verification on nearly half the dataset.

show abstract

A Purely Visual Re-ID Approach for Bumblebees (Bombus terrestris)

Borlinghaus

Tausch²,

Rettenberger

2023

Smart Agricultural Technology

View full text Add to dashboard Cite

Method for Whale Re-identification Based on Siamese Nets and Adversarial Training

Cited by 4 publications

References 27 publications

FIN-PRINT a fully-automated multi-stage deep-learning-based framework for the individual recognition of killer whales

FIN-PRINT a fully-automated multi-stage deep-learning-based framework for the individual recognition of killer whales

Finding Nemo’s Giant Cousin: Keypoint Matching for Robust Re-Identification of Giant Sunfish

A Purely Visual Re-ID Approach for Bumblebees (Bombus terrestris)

Contact Info

Product

Resources

About