Pointing and voicing in deictic expressions

Abstract

The present paper studies how, in deictic expressions, the temporal interdependency of speech and gesture is realized in the course of motor planning and execution. Two theoretical positions were compared. On the "interactive" view the temporal parameters of speech and gesture are claimed to be the result of feedback between the two systems throughout the phases of motor planning and execution. The alternative "ballistic" view, however, predicts that the two systems are independent during the phase of motor execution, the temporal parameters having been preestablished in the planning phase. In four experiments subjects were requested to indicate which of an array of referent lights was momentarily illuminated. This was done by pointing to the light and/or by using a deictic expression (this/that light). The temporal and spatial course of the pointing movement was automatically registered by means of a Selspot opto-electronic system. By analyzing the moments of gesture initiation and apex, and relating them to the moments of speech onset, it was possible to show that, for deictic expressions, the ballistic view is very nearly correct . 0 1985 Academic Press, IncThe general issue addressed in this article concerns the synchronization of speech and gesture. More specifically the aim is to investigate how the frequently noted interdependence of speech and gesture is realized in the course of motor planning and execution. Do the two systems operate interactively, in the sense that mutual adaptation takes place during the phase of motor execution, or do they rather operate in a ballistic or independent fashion in so far as

Do conversational hand gestures communicate?

“…An alternative view takes as its point of departure the fact that gestures and lexical affiliates are coordinated temporally; a finding that has been interpreted to indicate that the two behaviors have a common origin (Kendon, 1980(Kendon, , 1983MorrelSamuels & Krauss, 1991;Levelt, Richardson, & La Heij, 1985;McNeill, 1985McNeill, , 1987Schegloff, 1984). However, there is little consensus as to the locus of that common origin or the process by which gestures are transformed into articulate movements.…”

Asymmetries of the human social brain in the visual, auditory and chemical modalities

“…It has also been suggested that language evolved from manual gestures: the relationship between hand and mouth begun as ingestive movements, progressively adapted for communication (Gentilucci & Corballis 2006). Others posit that gesture and speech are two different communication systems, depending upon different brain structures (Levelt et al 1985;Hadar et al 1998). The localization evidence obtained so far points to a bilateral brain network underlying gesture perception, that however includes many of the known structures in the right hemisphere that have been shown to support the perception of biological motion per se: for instance, an fMRI study by Villarreal et al (2008) demonstrated an extensive, common network underlying the recognition of gestures consisting of the right pre-supplementary motor area, the right STS/ STG, the left inferior posterior parietal cortex, and bilateral superior posterior parietal cortex (PPc), precuneus, fusiform gyri, occipitotemporal regions and visual cortices.…”

The timing of head movements: The role of prosodic heads and edges

“…There is evidence that the scope of gestural movements typically finishes at the end of intonational phrases (Loehr, 2012;Shattuck-Hufnagel et al, 2010;see Krivokapić, 2014, for a review) and that listeners can automatically extract prosodic structure by using the temporal scope of manual beat gestures and thus use these gestural features disambiguating the syntactic structure (Guella€ ı et al, 2014). Interestingly, phrase boundaries seem to impact not only the ending point of a gesture movement, but also the timing of the distinct gesture phases in relation to speech landmarks (De Ruiter, 1998;Esteve-Gibert and Prieto, 2013;Krivokapić et al, 2015;Krivokapić et al, 2016;Levelt et al, 1985). EsteveGibert and Prieto (2013) observed that the movement pattern of the manual pointing gestures mimicked that of F0 movements.…”

“…Manual gestures are by far the most-studied gestures, beat and pointing manual movements traditionally receiving most of the researchers' attention (e.g., Kendon, 1980;Leonard and Cummins, 2011;Treffner et al, 2008, for beat gestures;De Ruiter, 1998;Levelt et al, 1985;RochetCapellan et al, 2008;Roustan and Dohen, 2010, for pointing gestures). Leonard and Cummins (2011) used a motion caption system to track hand gestures while participants were reading a short fable.…”