The painful face – Pain expression recognition using active appearance models

Abstract

Pain is typically assessed by patient self-report. Self-reported pain, however, is difficult to interpret and may be impaired or in some circumstances (i.e., young children and the severely ill) not even possible. To circumvent these problems behavioral scientists have identified reliable and valid facial indicators of pain. Hitherto, these methods have required manual measurement by highly skilled human observers. In this paper we explore an approach for automatically recognizing acute pain without the need for human observers. Specifically, our study was restricted to automatically detecting pain in adult patients with rotator cuff injuries. The system employed video input of the patients as they moved their affected and unaffected shoulder. Two types of ground truth were considered. Sequence-level ground truth consisted of Likert-type ratings by skilled observers. Frame-level ground truth was calculated from presence/absence and intensity of facial actions previously associated with pain. Active appearance models (AAM) were used to decouple shape and appearance in the digitized face images. Support vector machines (SVM) were compared for several representations from the AAM and of ground truth of varying granularity. We explored two questions pertinent to the construction, design and development of automatic pain detection systems. First, at what level (i.e., sequence- or frame-level) should datasets be labeled in order to obtain satisfactory automatic pain detection performance? Second, how important is it, at both levels of labeling, that we non-rigidly register the face?

Pain Level Detection From Facial Image Captured by Smartphone

“…That time combination of two or more classifiers provide greater accuracy. In some cases, active appearance models (AAM) are used to identify specific facial features associated with pain [16], [27]. To find a computationally inexpensive solution the Eigenface-based method was deployed [28].…”

“…Another idea with video sequences have come from Ashraf et al He collected all the video sequences of facial expressions and calculated the pain level [27]. UNBC-McMaster shoulder pain expression database was used here.…”

Spatiotemporal Facial Super-Pixels for Pain Detection

“…AUs have also been used in [4] for defining a rule-based system for detecting the pain/no-pain case. Active Appearance Models (AAM) have been used in [2] to decouple shape and appearance parameters from digitized facial images, while in [15] Multiple Instance Learning (MIL) has been used to handle training data by putting it into bags, which are labeled as either positive, if the bag contains a positive instance, or negative, if no positive instances exist in the bag. Then, a Bag of Words (BoW) approach is used for determining whether a set of frames contains pain or not.…”

Automatic Facial Expression Recognition

“…Their systems during the last decade have utilized Active Appearance Models for face registration, thus allowing them to estimate the locations of 68 different feature points which are then classified into an expression. It is noteworthy, however, that their group has recently augmented the feature vector of their expression recognizer with appearance-based features: Given an AAM fitted to the face, the feature vector then consists of the non-rigid shape parameters (geometric features), concatenated with the pixel values of the face after removing teh non-rigid shape variation by warping it back onto a canonical face model [53,2]. The combined feature vector is then classified by a support vector machine.…”