Two Ways to Facial Expression Recognition? Motor and Visual Information Have Different Effects on Facial Expression Recognition

Abstract

Motor-based theories of facial expression recognition propose that the visual perception of facial expression is aided by sensorimotor processes that are also used for the production of the same expression. Accordingly, sensorimotor and visual processes should provide congruent emotional information about a facial expression. Here, we report evidence that challenges this view. Specifically, the repeated execution of facial expressions has the opposite effect on the recognition of a subsequent facial expression than the repeated viewing of facial expressions. Moreover, the findings of the motor condition, but not of the visual condition, were correlated with a nonsensory condition in which participants imagined an emotional situation. These results can be well accounted for by the idea that facial expression recognition is not always mediated by motor processes but can also be recognized on visual information alone.

Perceiving emotions in visual stimuli: social verbal context facilitates emotion detection of words but not of faces

“…Whereas this might appear to contradict the view that cognition and emotional meaning is grounded in sensorimotor processes, there is research suggesting that facial simulation does not always relate to or occur in emotion processing tasks (e.g., Arnold and Winkielman 2019 ). Various routes to emotion processing have been shown to play a role in detecting and grasping emotional concepts (e.g., Arnold and Winkielman 2019 ; de la Rosa et al 2018 ; Stel 2016 ). In the present study such routes pertain to visual information, auditory information, semantic information as well as previous knowledge on emotion concepts.…”

“…Accurately detecting emotion-related information is an important ability for successful human interaction. Grounded in the embodied cognition view of social information processing (e.g., Arnold and Winkielman 2019 ; Niedenthal 2007 ; Niedenthal et al 2005 ), in numerous emotion processing studies, both written words as well as facial expressions have been shown to trigger a simulation process that sometimes is reflected in facial muscle activity (e.g., Foroni and Semin 2009 ; Niedenthal 2007 ). Especially the simulation of facial expressions involves cortical processing related to motor simulation of facial expressions, the posterior cingulate cortex, and medial temporal lobe structures (Schilbach et al 2008 ).…”

“…For example, the smiling muscle -zygomaticus major- tends to show more activity when people perceive positive emotional information, and less activity to negative emotional information. Similarly, the frowning muscle -corrugator supercilii- tends to become activated when people perceive negative information, while showing less activity to positively valenced information (e.g., Dimberg et al 2000 , 2002 ). This process is also referred to as facial simulation or mimicry.…”

“… 1 Six positive and six negative Dutch words previously used in a study by Foroni and Semin ( 2009 ), and 9 positive, 9 negative, and 18 neutral Dutch words from a study reporting their emotional meaning and perceived valence (Hermanns and De Houwer 1994 ) were selected. Lastly, selected words from a study reporting affective norms of English words (Bradley and Lang 1999 ) were translated into Dutch and pilot tested for the current study on their valence and emotionality.…”

The intensity of emotion: Altered motor simulation impairs processing of facial expressions in congenital facial palsy

“…Nevertheless, experimental evidence that newborns and infants mimic others' facial expressions and react to them accordingly with the emotions they convey, seems to contrast with this view; for instance, they react with avoidance and fear to negative expressions long before a sophisticated capacity for ToM has developed (Ruba, Meltzoff, & Repacholi, 2019), suggesting that the potential process underlying understanding others' emotions is more likely based on early, pre-wired mechanisms rather than on high-level cognitive skills (Preston & de Waal, 2002). An alternative mechanism to the ToM proposes that the recognition of others' emotions requires both visual analysis and 'sensorimotor simulation' (de la Rosa, Fademrecht, Bülthoff, Giese, & Curio, 2018;Paracampo, Pirruccio, Costa, Borgomaneri, & Avenanti, 2018) -an unconscious, covert imitation and automatic activation of the sensorimotor programs of the observed facial postures or movements (Goldman & Sripada, 2005;Ipser & Cook, 2016;Montgomery & Haxby, 2008;Niedenthal, Mermillod, Maringer, & Hess, 2010;Paracampo, Tidoni, Borgomaneri, Di Pellegrino, & Avenanti, 2017), suggesting that humans are 'cabled' with a neural mechanism that let them to almost literally resonate with others (Adolphs, 2009;Ferrari, Tramacere, Simpson, & Iriki, 2013;Gallese, 2016). This mechanism, in turn, would activate interconnected systems, including the limbic system, allowing observers to re-enact the others' affective states, and therefore, in last analysis, to understand them (Wood, Rychlowska, Korb, & Niedenthal, 2016;Tramacere, Pievani, & Ferrari, 2017).…”