Seeing first person changes gesture but saying first person does not

Abstract

Events with a motor action component (e.g., handling an object) tend to evoke gestures from the point of view of a character (character viewpoint, or CVPT) while events with a path component (moving through space) tend to evoke gestures from the point of view of an observer (observer viewpoint, or OVPT). Events that combine both components (e.g., rowing a boat across a lake) seem to evoke both types of gesture, but it is unclear why narrators use one or the other. We carry out two manipulations to explore whether gestural viewpoint can be manipulated. Participants read a series of stories and retold them in two conditions. In the image condition, story sentences were presented with images from either the actor’s perspective (actor version) or the observer’s perspective (observer version). In the linguistic condition, the same sentences were presented in either the second person (you…) or the third person point of view (h/she…). The second person led participants to use the first person (I) in retelling. Gestures produced during retelling were coded as CVPT or OVPT. Participants produced significantly more CVPT gestures after seeing images from the point of view of an actor, but the linguistic manipulation did not affect viewpoint in gesture. Neither manipulation affected overall gesture rate, or co-occurring speech. We relate these findings to frameworks in which motor action and mental imagery are linked to viewpoint in gesture.

Gesture as simulated action: Revisiting the framework

“…Whether a gesture is produced with a character or observer viewpoint thus depends on how the event is simulated, which may in turn depend on how the event was experienced and on what information about the event is particularly salient. For example, Parrill and Stec (2018) found that participants who saw a picture of an event from a first-person perspective were more likely to produce character viewpoint gestures about the event than participants who saw a picture of the same event from a third-person perspective. Moreover, certain kinds of events are particularly likely to be simulated from the point of view of a character.…”

The relationship between character viewpoint gesture and narrative structure in children

“…A next step could be to examine different types of CVPT gesture. CVPT gestures are not all equivalent (Parrill & Stec, 2018). A gesture with an actor's handshape is usually considered sufficient to code a gesture CVPT, but this handshape may reflect minimal perspective taking.…”

“…However, there is no direct evidence that CVPT gesture can be used to infer a first person point of view. Perhaps the closest is a study by Parrill and Stec (2018). In this study, participants read stories containing either second person pronouns ( you are …) or third person pronouns ( she is …), and then narrated the stories.…”

Construing events first-hand: Gesture viewpoints interact with speech to shape the attribution and memory of agency

“…Whereas many studies have focused on how iconic gestures objectively capture features of the external world, gesture can also reveal a speaker's subjective relationship to that world (Debreslioska et al, 2013;Hostetter & Alibali, 2019;Masson-Carro et al, 2016;McNeill, 1985McNeill, , 1992Parrill, 2010;Parrill & Stec, 2018). This means that in addition to enriching the content of spoken sentences, gestures may also shed light on how speakers see themselves with respect to what they are describing.…”

“…Given that English speakers use transitive and intransitive sentences to mark agentic roles in an event , it is likely that gestures also signal that relationship. Indeed, Fey Parrill (Parrill, 2010;Parrill & Stec, 2018) has shown that differences in gesture viewpoints align with linguistic differences in agentivity, with character viewpoint gestures going mostly with transitive sentences (e.g., saying "I dropped the vase," while gesturing two hands letting go of an object) and object viewpoint gestures going mostly with intransitive sentences (e.g., saying, "The vase dropped," while making a balled fist and moving it downward; see also Debreslioska et al, 2013). In this way, speakers can mark their own agentic relationship to actions and objects through multiple channels-through speech and gesture.…”