“I Know That You Know How I Feel”: Behavioral and Physiological Signals Demonstrate Emotional Attunement While Interacting with a Computer Simulating Emotional Intelligence

Balzarotti, Stefania; Piccini, Luca; Andreoni, Giuseppe; Ciceri, Maria Rita

doi:10.1007/s10919-014-0180-6

Cited by 19 publications

(12 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…I'm afraid I can't do that'' left movie viewers wondering if computers would one day be able to assume human emotions. While it remains to be seen if computers can indeed assume such qualities, research by Balzarotti et al (2014) reported here suggests that individuals may nevertheless believe that they do, and respond in predictable ways to affective human-computer interactions. When playing a game with a computer that simulated understanding of their emotional reactions while guiding them through game activities, participants reported a sense of awareness of interacting with an agent able to recognize their emotions and ''understand'' them.…”

mentioning

confidence: 75%

Nonverbal Behavior in the Digital Age: Meanings, Models, and Methods

Montepare

2014

J Nonverbal Behav

View full text Add to dashboard Cite

This Special Issue of the Journal of Nonverbal Behavior explores nonverbal issues in the digital age, with an emphasis how digital technologies provide researchers with new tools for exploring nonverbal components of interpersonal interactions in digital environments. In addition to showing a range of exciting applications, the featured articles also touch upon the challenges of new digital strategies.

show abstract

mentioning

confidence: 75%

Nonverbal Behavior in the Digital Age: Meanings, Models, and Methods

Montepare

2014

J Nonverbal Behav

View full text Add to dashboard Cite

show abstract

“…More generally, the current findings also need consideration in the portrayal of laughter by virtual agents. The adequate communication of emotions has been postulated a necessary condition of functioning human-virtual character interactions (see Niewiadomski et al, 2012 ; Balzarotti et al, 2014 ). Thus, if frowning alters the perception of a whole laughter expression in certain presentation modes, it can consequently corrupt the interaction with (or evaluation of) a virtual character.…”

Section: Discussionmentioning

confidence: 99%

Intense or malicious? The decoding of eyebrow-lowering frowning in laughter animations depends on the presentation mode

Hofmann

2014

Front. Psychol.

View full text Add to dashboard Cite

Joyful laughter is the only laughter type that has received sufficient validation in terms of morphology (i.e., face, voice). Still, it is unclear whether joyful laughter involves one prototypical facial-morphological configuration (Duchenne Display and mouth opening) to be decoded as such, or whether qualitatively distinct facial markers occur at different stages of laughter intensity. It was proposed that intense laughter goes along with eyebrow-lowering frowning, but in decoding studies of pictures, these “frowns” were associated with perceived maliciousness rather than higher intensity. Thus, two studies were conducted to investigate the influence of the presentation mode (static, dynamic) and eyebrow-lowering frowning on the perception of laughter animations of different intensity. In Study 1, participants (N = 110) were randomly assigned to two presentation modes (static pictures vs. dynamic videos) to watch animations of Duchenne laughter and laughter with added eyebrow-lowering frowning. Ratings on the intensity, valence, and contagiousness of the laughter were completed. In Study 2, participants (N = 55) saw both animation types in both presentation modes sequentially. Results confirmed that the static presentation lead to eyebrow-lowering frowning in intense laughter being perceived as more malicious, less intense, less benevolent, and less contagious compared to the dynamic presentation. This was replicated for maliciousness in Study 2, although participants could potentially infer the “frown” as a natural element of the laugh, as they had seen the video and the picture. Thus, a dynamic presentation is necessary for detecting graduating intensity markers in the joyfully laughing face. While this study focused on the decoding, future studies should investigate the encoding of frowning in laughter. This is important, as tools assessing facially expressed joy might need to account for laughter intensity markers that differ from the Duchenne Display.

show abstract

“…Nevertheless, some of the reviewed studies investigated product categories with low, unknown, or undefined emotional competence. Some articles (three studies) used computer games to evoke adults' (Rossi, 2013;Balzarotti et al, 2014) as well as children's (Mui et al, 2017) affective responses in order to investigate their behavior under social and/or situational contexts. Tutoring programs/software were used in several studies to evoke an affective response in order to study affective states during learning (four studies) or to develop products and/or software (two studies).…”

Section: Consumer Product-based Stimulimentioning

confidence: 99%

“…Likewise, facial electromyography (EMG) serves to measure and map the underlying electrical activity that is generated when muscles contract during discrete choicemaking (Rasch et al, 2015) while eye-tracking technology has been harnessed to study gaze behavior with respect to packaging, label and menu design, in-store consumer behavior, emotional responses, and eating disorders (Duerrschmid and Danner, 2018). In fact, neuromarketing techniques are often used in conjunction with FACS to provide a more holistic picture of the consumer affective experience (Cohn et al, 2002;Balzarotti et al, 2014;Lagast et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

The Facial Action Coding System for Characterization of Human Affective Response to Consumer Product-Based Stimuli: A Systematic Review

et al. 2020

View full text Add to dashboard Cite

To characterize human emotions, researchers have increasingly utilized Automatic Facial Expression Analysis (AFEA), which automates the Facial Action Coding System (FACS) and translates the facial muscular positioning into the basic universal emotions. There is broad interest in the application of FACS for assessing consumer expressions as an indication of emotions to consumer product-stimuli. However, the translation of FACS to characterization of emotions is elusive in the literature. The aim of this systematic review is to give an overview of how FACS has been used to investigate human emotional behavior to consumer product-based stimuli. The search was limited to studies published in English after 1978, conducted on humans, using FACS or its action units to investigate affect, where emotional response is elicited by consumer product-based stimuli evoking at least one of the five senses. The search resulted in an initial total of 1,935 records, of which 55 studies were extracted and categorized based on the outcomes of interest including (i) method of FACS implementation; (ii) purpose of study; (iii) consumer product-based stimuli used; and (iv) measures of affect validation. Most studies implemented FACS manually (73%) to develop products and/or software (20%) and used consumer product-based stimuli that had known and/or defined capacity to evoke a particular affective response, such as films and/or movie clips (20%); minimal attention was paid to consumer products with low levels of emotional competence or with unknown affective impact. The vast majority of studies (53%) did not validate FACS-determined affect and, of the validation measures that were used, most tended to be discontinuous in nature and only captured affect as it holistically related to an experience. This review illuminated some inconsistencies in how FACS is carried out as well as how emotional response is inferred from facial muscle activation. This may prompt researchers to consider measuring the total consumer experience by employing a variety of methodologies in addition to FACS and its emotion-based interpretation guide. Such strategies may better conceptualize consumers' experience with products of low, unknown, and/or undefined capacity to evoke an affective response such as product prototypes, line extensions, etc.

show abstract

“I Know That You Know How I Feel”: Behavioral and Physiological Signals Demonstrate Emotional Attunement While Interacting with a Computer Simulating Emotional Intelligence

Cited by 19 publications

References 44 publications

Nonverbal Behavior in the Digital Age: Meanings, Models, and Methods

Nonverbal Behavior in the Digital Age: Meanings, Models, and Methods

Intense or malicious? The decoding of eyebrow-lowering frowning in laughter animations depends on the presentation mode

The Facial Action Coding System for Characterization of Human Affective Response to Consumer Product-Based Stimuli: A Systematic Review

Contact Info

Product

Resources

About