Classifying pretended and evoked facial expressions of positive and negative affective states using infrared measurement of skin temperature

Abstract

Earlier researchers were able to extract transient facial thermal features from thermal infrared images (TIRIs) to make binary distinction between the expressions of affective states. However, affective human-computer interaction might require machines to distinguish between the subtle facial expressions of affective states. This work, for the first time, attempts to use the transient facial thermal features to recognise a much wider range of facial expressions. A database of 324 time-sequential, visible-spectrum and thermal facial images was developed representing different facial expressions from 23 participants in different situations. A novel facial thermal feature extraction, selection and classification approach was developed and invoked on various Gaussian mixture models constructed using: neutral and pretended happy and sad faces; faces with multiple positive and negative facial expressions; faces with neutral and six (pretended) basic facial expressions; and faces with evoked happiness, sadness, disgust and anger. This work demonstrates that (1) infrared imaging can be used to observe affective-state-specific thermal variations on the face; (2) pixel-grey level analysis of TIRIs can help localise significant facial thermal feature points along the major facial muscles; and (3) cluster-analytic classification of transient thermal features can help distinguish between the facial expressions of affective states in an optimized eigenspace of input thermal feature vectors. The observed classification results also exhibited influence of a Gaussian mixture model's structure on classifier-performance. The work also unveiled some pertinent aspects of future research on the use of facial thermal features in automated facial expression classification and affect recognition.

Detecting emotional stress during typing task with time pressure

“…To evoke users' stress in the experiments, there are some methods which are widely adopted, which include mental arithmetic, N-back number recall, time pressure, reading aloud, viewing affective picture or video, emotive text reading and story telling (see [6]- [9]). Among these methods, some are very useful to enable the job demands to be quantified or measured, for example mental arithmetic, N-back number recall and time pressure.…”

“…To classify stress versus non-stress conditions, besides the traditional social science research methods, the most common approaches are physiological measurements [8], [12]- [17] and www.conference.thesai.orgfacial expressions recognition [9], [18]- [23]. Although both physiological measures and facial expression recognition have high accuracy rates, but the assessments could be obtrusive, requiring additional equipment (which can be costly), and are often labour or computationally intensive [6], [24].…”

Non-intrusive car driver's emotion recognition using thermal camera

“…Such issues were encountered during the creation of TIVs. Another issue encountered with the creation of TIVs was due to the inconsistency of size of face image of different volunteers to that of actual square grids mentioned in [24] and [34]. To overcome such issues each face image was resized.…”

“…The technique proposed by Khan et al [24] showed 66.28% of successful classification. During cross validation test and 56% of successful classification rate was reported during a person independent test [34]. If this technique is adapted into car/automobile the performance of the classifier could worsen as the input image consists of multiple faces.…”

The Affectibility Concept in Systems for Learning Contexts

“…Among the "informational" methods are the proposals based on measurement of biophysiological responses and their automatic translation into words that define the emotions measured (e.g. [26] and [31]). …”