Emotion Recognition in Horses with Convolutional Neural Networks

Abstract

Creating intelligent systems capable of recognizing emotions is a difficult task, especially when looking at emotions in animals. This paper describes the process of designing a “proof of concept” system to recognize emotions in horses. This system is formed by two elements, a detector and a model. The detector is a fast region-based convolutional neural network that detects horses in an image. The model is a convolutional neural network that predicts the emotions of those horses. These two elements were trained with multiple images of horses until they achieved high accuracy in their tasks. In total, 400 images of horses were collected and labeled to train both the detector and the model while 40 were used to validate the system. Once the two components were validated, they were combined into a testable system that would detect equine emotions based on established behavioral ethograms indicating emotional affect through the head, neck, ear, muzzle, and eye position. The system showed an accuracy of 80% on the validation set, demonstrating that it is possible to predict emotions in animals using autonomous intelligent systems. Such a system has multiple applications including further studies in the growing field of animal emotions as well as in the veterinary field to determine the physical welfare of horses or other livestock.

Going Deeper than Tracking: a Survey of Computer-Vision Based Recognition of Animal Pain and Affective States

“…For each of these stages, we classify the methods and techniques applied in these [76] unknown or naturally occurring face + Lencioni et al [77] Horses pain surgical castration face + Hummel et al [38] unknown or induced pain face + Broomé et al [78] induced pain body and face + Broomé et al [79] induced pain body and face + Rashid et al [80] induced pain body + Reulke et al [81] vet. procedure body -Corujo et al [82] emotion unknown body and face + Li et al [83] --face ---Feightelstein et al [84] Cats pain vet. procedure face +…”

“…The dataset used in [38] is composed from several sources: a clinical study, where pain was experimentally induced, images taken at a home housing older horses, and images provided by horse owners. In [82], the data is collected from different private sources where the horse and context of the photo was familiar, guiding annotation. However, the state annotation was performed by laymen.…”

“…The convolutional neural network (CNN) is the most popular type of deep model used in the works surveyed in this article. Examples of used CNN architectures include ResNet50 [74,82,84,89], InceptionV3 [73,74] and AlexNet [88]. One work addressing dog emotion [89] compared a CNN (ResNet50) to a Vision Transformer, ViT [119], a model fully based on attention mechanisms instead of convolutions, finding the latter to perform better.…”

“…• #cl: number of classes used for classification. In emotion recognition, the classes correspond to the emotions studied (e.g., relaxed/curious in [82], or happy in [88,90] ). In pain recognition, there is binary (pain/no pain) or three-level classification.…”

“…In datasets of this type, however, there is greater risk for bias and noise, introduced by data collected under unknown conditions. Datasets from private sources [38,82] or from the authors own clinical trials [73,77], on the other hand, typically involve a smaller number of individual animals, meaning that the risk is higher for the same animal with a similar expression to be present both in the training and testing set. In such cases the cross-validation method leave-one-animal-out is highly recommended, which also naturally enforces subject-exclusivity.…”