Chimpanzee face recognition from videos in the wild using deep learning

Abstract

Video recording is now ubiquitous in the study of animal behavior, but its analysis on a large scale is prohibited by the time and resources needed to manually process large volumes of data. We present a deep convolutional neural network (CNN) approach that provides a fully automated pipeline for face detection, tracking, and recognition of wild chimpanzees from long-term video records. In a 14-year dataset yielding 10 million face images from 23 individuals over 50 hours of footage, we obtained an overall accuracy of 92.5% for identity recognition and 96.2% for sex recognition. Using the identified faces, we generated co-occurrence matrices to trace changes in the social network structure of an aging population. The tools we developed enable easy processing and annotation of video datasets, including those from other species. Such automated analysis unveils the future potential of large-scale longitudinal video archives to address fundamental questions in behavior and conservation.

A Review of Face Recognition Technology

“…Recognizing the identity of people in video accurately and quickly is very important for video search and video surveillance. Schofield et al proposed a deep convolution neural network method, which could automatically detect, track and record human faces in video, and could be used to study the animal behavior [33] [34].…”

Jokro: The death of a wild infant chimpanzee from respiratory disease

“…However, our field experiments on tool use in Bossou provided an ideal opportunity to monitor the health and behavior of wild chimpanzees routinely: at regular intervals, in the same place, and under the same conditions. My colleagues and I have been running field experiments on stone tool use by wild chimpanzees at a designated site, Bossou's "outdoor laboratory", since 1988 (Matsuzawa 1994; Inoue-Nakamura and Matsuzawa 1997;Biro et al 2003;Schofield et al 2019). In this natural clearing in the forest, located on top of a sacred hill, experimental objects (nuts and stones) are provided by experimenters, and chimpanzees crack nuts using stone hammers and anvils.…”

Applying Deep Learning to Specific Learning Disorder Screening

“…Deep learning algorithms are more accurate than human beings in many visual tasks such as strategic board games, human and chimpanzee facial recognition, plant disease identification, and object recognition (Esteva et al, 2017;Ferentinos et al, 2018;Schofield et al, 2019). In addition, deep learning algorithms perform better than humans in medical diagnosis based on visual data such as skin cancer classification, breast cancer screening, and pneumonia detection (Esteva et al, 2017;McKinney et al,2020;Rajpurkar et al, 2017).…”