Memory for Moving Faces: The Interplay of Two Recognition Systems

Abstract

The human face is a captivating stimulus, even when it is stationary. In motion, however, the face comes to life and offers us a myriad of information about the intent and personality of its owner. Through facial movements such as expressions, we can gauge a person's current state of mind. By perceiving the movements of the mouth as a friend speaks, a conversation becomes more intelligible in a noisy environment. Through the rigid movement of the head and the direction of eye gaze, we can follow another person's focus of attention in a crowded room. The amount and diversity of social information that can be conveyed by a face assures its place as a central focal object in any scene. Beyond the rich communication signals that we perceive in facial expressions, head orientation, eye gaze, and facial speech motions, it is also pertinent to ask whether the movements of a face help us to remember a person. The answer to this question can potentially advance our understanding of how the complex tasks we perform with faces, including those having to do with social interaction and memory, co-exist in a neural processing network. It can also shed light on the computational processes we use to extract recognition cues from the steady stream of facial movements that carry social meanings.Several years ago, we proposed a psychological and neural framework for understanding the contribution of motion to face recognition (O'Toole, Roark, and Abdi, 2002;Roark, Barrett, Spence, Abdi, and O'Toole, 2003). At the time,

On the facilitative effects of face motion on face recognition and its development

“…It should be mentioned that the two prosopagnosia studies above only included congenitally prosopagnosic patients, rather than acquired prosopagnosic patients. As noted in O'Toole and Roark ( 2010 ), the observed dissociation between static and dynamic facial information processing reflects the possibility that the ability to process facial movements has been developed to compensate for impaired static face processing. One study found that an acquired prosopagnosic patient exhibited deficits in abilities required to recognize faces from both static and dynamic facial information (Lander et al, 2004 ).…”

“…Since the publication of this initial finding, other researchers have consistently observed beneficial effects for facial movement in recognizing famous faces, familiar faces, and unfamiliar faces (Pike et al, 1997 ; Lander et al, 1999 , 2004 ; Lander and Bruce, 2000 , 2003 , 2004 ; Bruce et al, 2001 ; Thornton and Kourtzi, 2002 ; Pilz et al, 2006 ; Roark et al, 2006 ; Lander and Davies, 2007 ; O'Toole et al, 2011 ; Arnold and Siéroff, 2012 ). Based on this body of work, researchers have proposed two mechanisms regarding facial motion's enhancement of recognition: one suggesting that facial movement provides idiosyncratic facial information in addition to static facial information (the supplementary information hypothesis), and the other suggesting that facial movements assist in producing a more robust and flexible three-dimensional face representation in learning new faces, thus improving face recognition (the representation enhancement hypothesis: O'Toole et al, 2002 ; O'Toole and Roark, 2010 ). In the following sections, we will discuss these two hypotheses and their supporting evidence.…”

“…Studies have shown that learning new faces in motion leads to better recognition performance than learning the face from a static image. This facial motion facilitation effect in learning new faces suggests that facial movements assist in forming a robust face representation, which in turn leads to enhanced face recognition performance (the representation enhancement hypothesis, O'Toole et al, 2002 ; O'Toole and Roark, 2010 ).…”

“…Support for the representation enhancement hypothesis is shown when faces are recognized more accurately when they are learned in motion rather than from single or multiple static face images (O'Toole and Roark, 2010 ). For example, Pike et al ( 1997 ) presented participants with rigidly moving face videos (i.e., head rotation), or multiple static face pictures extracted from the videos.…”

“…In a recent revision of the representation enhancement hypothesis, O'Toole and Roark ( 2010 ) posited that the representation enhanced by facial movement might not be limited to the static components of face representation. Facial movements may also enhance a general dynamic face representation, which could only be accessed with facial movements.…”