Automated Identification of Individual Great White Sharks from Unrestricted Fin Imagery

Abstract

Introduction. Recognising individuals repeatedly over time is a basic requirement for field-based ecology and related life sciences [5,6]. In this paper we propose a visual identification approach for great white shark fins as outlined in Figure 1, one that is applicable to unconstrained fin imagery and fully automates the pipeline from feature extraction to matching of identities. We pose the associated vision task as a fine-grained, multiinstance classification problem for flexible, smooth and partly occluded object parts. To the best of our knowledge, the proposed system is the first fully automated contour-based animal biometrics system. Contour Stroke Object Model. A key technical contribution of the paper is a contour stroke model trained for fin detection. It combines a partitioning of ultrametric contour maps (UCM; generated via [3,4]) with normal descriptors, dense local features and a random-forest regressor. UCM region boundaries are partitioned at keypoints detected as local maxima of the Difference of Gaussian (DoG) function D(u, σ ) detailed by Zhang et al. [10]. We generate fin candidates as contour strokes by sampling the region contour between every permutation of keypoint pairs. Each stroke is then described by a 160 dimensional feature vector consisting of two components: the first is a bag of opponentSIFT [9] visual words, the second describes boundary shape using a histogram of boundary normals. A random forest regressor is finally trained to predict the fin-like quality of contour stroke candidates. Figure 2 illustrates fins detected by this strategy. In common with the segmentation stage of Arandjelovic and Zisserman's 'Bag of Boundaries' pipeline [2], our contour stroke model separates objects in natural images and against cluttered backgrounds, but differs in that it exploits species characteristic shape encoded in the open contours, to specifically and explicitly detect fin parts.Biometric Encoding: [1, 2] perform smooth object recognition using semi-local boundary normal and HoG+occupancy representations, respectively. These allow efficient and robust matching, but their encoding of inter-class variance will always be sub-maximal. Meanwhile, the semiautomated fin recognition system DARWIN [8] uses a global 2D Cartesian representation. This encoding maximises inter-class variance, but is sensitive to partial occlusions and detection errors. By contrast we utilise semi-local and global shape descriptions in a vector-based combinatorial encoding strategy, one that enables efficient and accurate individual recognition while being robust to noisy, partially occluded input.As with generating contour strokes, we combinatorially sample fin boundaries between pairs of keypoints detected using D(u, σ ), reparameterised for fine-grained feature detection. We then scale-normalise the resulting contour subsections and encode their shape over various spectral scales. Denoting the set of descriptors for a query object at scale j as D

Automated Visual Fin Identification of Individual Great White Sharks

“…Overall, our contour stroke model for fin detection combined with a combinatorial biometric contour encoding proves suitable for the task of individual fin identification. For DoG N , as reported in Hughes and Burghardt (2015b) for one-shot-learning, of the 2371 query instances presented to the system, a particular shark is correctly identified with a mAP of 0.79. Figure 16 illustrates such examples of fin matches.…”

“…In this paper we will focus on contour information of textureless objects as biometric entities instead. In specific, we propose a visual identification approach for great white shark fins as schematically outlined in Figure 1, one that extends work in Hughes and Burghardt (2015b) and is applicable to unconstrained fin imagery. We perform a coarse and a fine-grained recognition task.…”

Affinity Matting for Pixel-accurate Fin Shape Recovery from Great White Shark Imagery

“…Overall, affinity mattes proved capable of localising ground truth contours to within-a-pixel. The method can be used as part of an ID system for great white sharks [7]. …”

“…Whenever animals carry individually unique visual markings and an approach for imaging these efficiently is available, biometric computer vision provides an option for non-invasive, partly or fully automated identification [9,12,14,15]. Individual great white sharks, for instance, can be fully automatically re-identified if one can photograph and match silhouettes of their dorsal fin [7], but this requires a precise (and fully automatic) extraction of boundaries as a precursor to matching. State-of-the-art image segmentation frameworks [1,2] may be used as locally coarse boundary detectors, but they rarely provide the levels of fine grained segmentation accuracy required for identification, particularly when applied at image resolutions optimised for efficiency 1 .…”

Automated face detection for occurrence and occupancy estimation in chimpanzees

“…Kühl and Burghardt () defined animal biometrics as the utilization of phenotypic characteristics that can identify species and in some scenarios even individuals, by exploiting body morphologies, coat patterns, and general appearance, vocalizations, or behaviors. Based on phenotypic observations and distinct animal characteristics, biometric software has helped to identify individual elephants from ear nicks (Ardovini, Cinque, & Sangineto, ), dolphins from dorsal fin shapes (Araabi, Kehtarnavaz, McKinney, Hillman, & Würsig, ), zebras from stripe patterns (Lahiri, Tantipathananandh, Warungu, Rubenstein, & Berger‐Wolf, ), great white sharks from dorsal fin shape (Hughes & Burghardt, ), and great apes from facial characteristics (Ernst & Küblbeck, ; Loos & Ernst, , ).…”