The Facial Action Coding System for Characterization of Human Affective Response to Consumer Product-Based Stimuli: A Systematic Review

Abstract

To characterize human emotions, researchers have increasingly utilized Automatic Facial Expression Analysis (AFEA), which automates the Facial Action Coding System (FACS) and translates the facial muscular positioning into the basic universal emotions. There is broad interest in the application of FACS for assessing consumer expressions as an indication of emotions to consumer product-stimuli. However, the translation of FACS to characterization of emotions is elusive in the literature. The aim of this systematic review is to give an overview of how FACS has been used to investigate human emotional behavior to consumer product-based stimuli. The search was limited to studies published in English after 1978, conducted on humans, using FACS or its action units to investigate affect, where emotional response is elicited by consumer product-based stimuli evoking at least one of the five senses. The search resulted in an initial total of 1,935 records, of which 55 studies were extracted and categorized based on the outcomes of interest including (i) method of FACS implementation; (ii) purpose of study; (iii) consumer product-based stimuli used; and (iv) measures of affect validation. Most studies implemented FACS manually (73%) to develop products and/or software (20%) and used consumer product-based stimuli that had known and/or defined capacity to evoke a particular affective response, such as films and/or movie clips (20%); minimal attention was paid to consumer products with low levels of emotional competence or with unknown affective impact. The vast majority of studies (53%) did not validate FACS-determined affect and, of the validation measures that were used, most tended to be discontinuous in nature and only captured affect as it holistically related to an experience. This review illuminated some inconsistencies in how FACS is carried out as well as how emotional response is inferred from facial muscle activation. This may prompt researchers to consider measuring the total consumer experience by employing a variety of methodologies in addition to FACS and its emotion-based interpretation guide. Such strategies may better conceptualize consumers' experience with products of low, unknown, and/or undefined capacity to evoke an affective response such as product prototypes, line extensions, etc.

Global-local attention for emotion recognition

“…However, our affective displays in reality are much more complicated and subtle compared to the simplicity of these universal emotions. To represent the complexity of the emotional spectrum, many approaches were proposed such as the Facial Action Coding System [8], where all facial actions are described in terms of Action Units (AUs); or dimensional models [46], where affection is quantified by values chosen over continuous emotional scales like valence and arousal. Nevertheless, those models which use discrete affections are the most popular in automatic emotion recognition task because they are easier to interpret and more intuitive to human.…”

Caught in the Act: Detecting Respondent Deceit and Disinterest in On-Line Surveys. A Case Study Using Facial Expression Analysis

“…The computer webcam recorded the respondent face during each 30–40 sec PSA. These recordings were uploaded to iMotions ® for analysis with the validated AFFDEX automated facial coding engine (Clark et al, 2020; De Lemos, 2007; Ekman & Friesen, 1978; Ekman, 1997; Stöckli et al, 2017). In this paper, “attention,” a measure of head position (pitch, yaw, and roll) relative to the camera, and 20 specific facial expressions (what the software algorithm classifies as, for example, “cheek raise,” “lip curl”) are used.…”

“…Researchers at a large southeastern university were awarded a grant from the state’s anti-tobacco agency to explore augmenting opinion-based surveys with data collected using biometric sensors (Al-Turk et al, 2018; Huseynov et al, 2019; Venkatraman et al, 2015; Cartocci et al, 2017). Our goal was to determine the value added by physiological neuro-metric tools (galvanic skin response, eye tracking, and facial expression analysis (FEA), (Clark et al, 2020; Hamelin et al, 2017; Joanne et al, 2019; Lewinski et al, 2014; Kong et al, 2020; Schmälzle & Meshi, 2020) to on-line surveys that use subjective measures for message pretesting.…”

An Online Experiment During COVID-19: Testing the Influences of Autonomy Support Toward Emotions and Academic Persistence

“…Facial expressions have long been used to indicate emotions and stayed central in emotion studies (Tomkins and McCarter, 1964;Russell, 1994;Ruba and Repacholi, 2020). The accuracy of using facial expressions to identify emotions has been justified through many ways, for instance, self-report instruments (Matsumoto, 1987;Matsumoto et al, 2000) and facial coding systems (Ekman et al, 1980;Clark et al, 2020;Rosenberg and Ekman, 2020). There is debate around the universality of facial expression.…”