Similarity Learning Networks for Animal Individual Re-Identification - Beyond the Capabilities of a Human Observer

Abstract

Deep learning has become the standard methodology to approach computer vision tasks when a large amount of labeled data is available. One area where traditional deep learning approaches fail to perform is one-shot learning tasks where a model must make accurate classifications after seeing only one example image. Here, we measure the capabilities of five Siamese similarity comparison networks based on the AlexNet, VGG-19, DenseNet201, MobileNetV2, and InceptionV3 architectures considering the challenging one-shot learning task of animal re-identification. We consider five data sets corresponding to five different species: humans, chimpanzees, humpback whales, fruit flies, and octopus, each with their own unique set of challenges. Using a five-fold validation split, we demonstrate that each network structure was able to successfully re-identify animal individuals, with DenseNet201 performing optimally with 89.7, 75.5, 61.4, 79.3 and 92.2 percentage accuracy on the human, chimpanzee, humpback whale, fruit fly, and octopus data sets respectively-without any species-specific modifications. Our results demonstrate that similarity comparison networks can achieve accuracies beyond humanlevel performance for the task of animal re-identification. The ability of a researcher to re-identify an animal individual upon re-encounter is fundamental for addressing a broad range of questions in the study of population dynamics and community/behavioural ecology. Our expectation is that similarity comparison networks are the beginning of a major trend that could stand to revolutionize animal re-identification from camera trap data.

Computer vision, machine learning, and the promise of phenomics in ecology and evolutionary biology

“…Deep learning algorithms are a family of machine learning methods based on artificial neural networks that "learn" what constitutes the object of interest during the training phase (LeCun et al, 2015;Heaton, 2020) . With sufficient training using labeled images (and in some cases unlabelled imagessee Box 2), deep learning-powered object detection algorithms can be highly accurate and often greatly outperform pre-existing object recognition methods (Krizhevsky et al, 2012;Alom et al, 2018) -in some cases even human experts, for example, when identifying species (Buetti-Dinh et al, 2019;Valan et al, 2019;Schneider et al, 2020b) . Each of these approaches has advantages and limitations, which mostly depend on the noise level within the images, the size of the dataset, and the availability of computational resources (see section "Practical considerations for CV" and Fig.…”

Automated facial recognition for wildlife that lack unique markings: A deep learning approach for brown bears

“…Brust et al 2017 testing individuals. We tested this latter mode to examine how our network would perform creating embeddings for bears not previously "seen," which is relevant to ecological application (Schneider et al, 2020). Retraining a classifier to include an option to designate new individuals (sensu Deb et al, 2018) would further support the automated use of this software in wildlife research and monitoring.…”

“…Face encoding forms the core process that facilitates facial recognition in the pipeline. It uses a similarity metric (Schneider et al, 2020) to learn a function that maps an input image (bear face chip) into a target space (Chopra et al, 2005). The metric loss function (Dlib toolkit: King, 2009) drives the similarity metric to be small for face chips of the same bear and large for face chips from different bears.…”

Revisiting animal photo-identification using deep metric learning and network analysis

“…Solving this problem was particularly challenging because of the size of our data set. Previous studies on animal re-identification with CNN indeed relied on a higher number of photographs per individuals (Schneider et al ., 2020; Ferreira et al ., 2020). In our case, we had to train the CNN with a few images per individuals only (see Snell et al ., 2017, on few shot learning methods) shot in the field with contrasting environmental and light conditions.…”

“…unknown individuals. However, despite the availability of proven and efficient techniques (Zheng et al ., 2016), and several successful attempts to apply the method to non-human species (Körschens et al ., 2018; Hansen et al ., 2018; Moskvyak et al ., 2019; Bouma et al ., 2019; Schofield et al ., 2019; He et al ., 2019; Bogucki et al ., 2019; Schneider et al ., 2020; Chen et al ., 2020; Ferreira et al ., 2020), re-identification remains a challenging task when applied to animals in the wild where re-observations are limited in number to train the model satisfactorily sensu largo (Schneider et al ., 2019).…”