Towards Machine Recognition of Facial Expressions of Pain in Horses

Abstract

Automated recognition of human facial expressions of pain and emotions is to a certain degree a solved problem, using approaches based on computer vision and machine learning. However, the application of such methods to horses has proven difficult. Major barriers are the lack of sufficiently large, annotated databases for horses and difficulties in obtaining correct classifications of pain because horses are non-verbal. This review describes our work to overcome these barriers, using two different approaches. One involves the use of a manual, but relatively objective, classification system for facial activity (Facial Action Coding System), where data are analyzed for pain expressions after coding using machine learning principles. We have devised tools that can aid manual labeling by identifying the faces and facial keypoints of horses. This approach provides promising results in the automated recognition of facial action units from images. The second approach, recurrent neural network end-to-end learning, requires less extraction of features and representations from the video but instead depends on large volumes of video data with ground truth. Our preliminary results suggest clearly that dynamics are important for pain recognition and show that combinations of recurrent neural networks can classify experimental pain in a small number of horses better than human raters.

Automated Recognition Of Pain In Cats

“…The results of the study on data labeled according to pain induction perform better compared to a veterinary expert baseline, i.e., four veterinarians with expert training in recognizing equine pain classifying 51 five second-clips from the dataset. A recent review by Andersen et al 42 provided a detailed comparison in the context of horse pain detection between approaches based on automatic detection of AUs, and deep learning models that are trained on raw videos of horses with known true pain status, both of which present promising results. Blumrosen and colleagues 43 proposed a different approach to overcome the difficulties in creating human-annotated datasets of facial expressions in non-human primates (NHP).…”

Pain assessment in horses using automatic facial expression recognition through deep learning-based modeling

“…Automated pain assessment has been evaluated in many animal species such as mice, sheep, and even humans [ 15 , 20 , 21 ]. A recent article addressed the complexities of assessing pain on horses through automatic recognition systems [ 22 ]. In addition, this resource could also be very useful as a way to educate students using a more visual and practical approach to pain recognition in horses [ 15 ].…”

Current Advances in Assessment of Dog’s Emotions, Facial Expressions, and Their Use for Clinical Recognition of Pain

“…The facial nerve innervates and regulates the movement of the face through the cutaneous muscle of the neck (platysma), the zygomaticus, the buccinator, and the mentalis. This cranial nerve has both motor and sensory fibers afferents from the parasympathetic nervous system, regulated by the peripheral nervous system [ 52 , 53 ].…”

“…The facial nerve also leaves the skull and passes superficially to the parotid gland to originate its terminal branches in the cervical limits (temporal, zygomatic, buccal, marginal of the mandible, and cervical). These branches finally disperse and innervate the muscles of facial expression or facial mimicry [ 52 , 55 ].…”

“…Further studies in dogs also require the development and implementation of techniques and tools currently applied in other species, such as artificial intelligence and computer vision systems for the recognition of pain gestures in humans and equines [ 158 ], or the automatic detection of Facial Units in sheep, a technique with up to 67% accuracy [ 52 ]. Likewise, cameras or visual sensors, global positioning systems (GPS), electrocardiography, electroencephalography [ 159 ], and infrared thermography [ 160 , 161 ] are alternatives to evaluate emotions and motor responses in dogs comprehensively.…”