Synchronization of speech and gesture: Evidence for interaction in action.

Chu, Mingyuan; Hagoort, Peter

doi:10.1037/a0036281

Cited by 45 publications

(55 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been clearly shown that the interaction between gestures and speech can occur during both their planning and execution phases (e.g., Chu & Hagoort, 2014;Kita & Özyürek, 2003).…”

Section: Comparison Of the Action Generation Hypothesis With Other Hymentioning

confidence: 99%

Co-thought and co-speech gestures are generated by the same action generation process.

Chu

Kita

2016

Journal of Experimental Psychology: Learning, Memory, and Cogni

Self Cite

View full text Add to dashboard Cite

People spontaneously gesture when they speak (co-speech gestures) and when they solve problems silently (co-thought gestures). In this study, we first explored the relationship between these 2 types of gestures and found that individuals who produced co-thought gestures more frequently also produced co-speech gestures more frequently (Experiments 1 and 2). This suggests that the 2 types of gestures are generated from the same process. We then investigated whether both types of gestures can be generated from the representational use of the action generation process that also generates purposeful actions that have a direct physical impact on the world, such as manipulating an object or locomotion (the action generation hypothesis). To this end, we examined the effect of object affordances on the production of both types of gestures (Experiments 3 and 4). We found that individuals produced co-thought and co-speech gestures more often when the stimulus objects afforded action (objects with a smooth surface) than when they did not (objects with a spiky surface). These results support the action generation hypothesis for representational gestures. However, our findings are incompatible with the hypothesis that co-speech representational gestures are solely generated from the speech production process (the speech production hypothesis).

show abstract

“…It has been clearly shown that the interaction between gestures and speech can occur during both their planning and execution phases (e.g., Chu & Hagoort, 2014;Kita & Özyürek, 2003).…”

Section: Comparison Of the Action Generation Hypothesis With Other Hymentioning

confidence: 99%

Co-thought and co-speech gestures are generated by the same action generation process.

Chu

Kita

2016

Journal of Experimental Psychology: Learning, Memory, and Cogni

Self Cite

View full text Add to dashboard Cite

show abstract

“…Implications for models of speech and gesture production Over the years, various models of speech and gesture production have been proposed, including Krauss, Chen and Gottesman's (2000) Process model, Kita and Özyürek's (2003) Interface model, de Ruiter's (2000) Sketch model, and McNeill and Duncan's (2000) Growth Point theory (see e.g., Chu & Hagoort, 2014;Hostetter & Alibali, 2008;Wagner, Malisz, & Kopp, 2014, for recent comparisons and discussion). These models all seek to describe how speakers produce multimodal utterances and are concerned with issues such as the timing and integration of gesture and speech, and the role that gestures play in communication.…”

Section: On the Effects Of Visibilitymentioning

confidence: 99%

“…Finally, in the third stage, the utterance plan is phonologically encoded and articulated, resulting in overt, auditory speech. Models of gesture production typically involve two stages: a Motor Planning stage, sometimes referred to as the Gesture Planner or the Action Generator, during which the motor instructions are produced, and a Motor Execution stage, during which these programs are executed, resulting in overt, visible gestures (Chu & Hagoort, 2014;Wagner et al, 2014).…”

Section: On the Effects Of Visibilitymentioning

confidence: 99%

Reduction in gesture during the production of repeated references

Hoetjes

Koolen

Goudbeek

et al. 2015

Journal of Memory and Language

View full text Add to dashboard Cite

a b s t r a c tIn dialogue, repeated references contain fewer words (which are also acoustically reduced) and fewer gestures than initial ones. In this paper, we describe three experiments studying to what extent gesture reduction is comparable to other forms of linguistic reduction. Since previous studies showed conflicting findings for gesture rate, we systematically compare two measures of gesture rate: gesture rate per word and per semantic attribute (Experiment I). In addition, we ask whether repetition impacts the form of gestures, by manual annotation of a number of features (Experiment I), by studying gradient differences using a judgment test (Experiment II), and by investigating how effective initial and repeated gestures are at communicating information (Experiment III). The results revealed no reduction in terms of gesture rate per word, but a U-shaped reduction pattern for gesture rate per attribute. Gesture annotation showed no reliable effects of repetition on gesture form, yet participants judged gestures from repeated references as less precise than those from initial ones. Despite this gradient reduction, gestures from initial and repeated references were equally successful in communicating information. Besides effects of repetition, we found systematic effects of visibility on gesture production, with more, longer, larger and more communicative gestures when participants could see each other. We discuss the implications of our findings for gesture research and for models of speech and gesture production.

show abstract

“…The finding suggests that the fine motor articulation required for grasping is processed similarly by both hand and mouth in humans, thus they tend to complement each other. In fact, so tightly are the two motor systems entwined that when either gesture or speech is disrupted the other becomes delayed (Chu & Hagoort, 2014).…”

Section: Introductionmentioning

confidence: 99%

Slip of the tongue: Implications for evolution and language development

Forrester

2015

Cognition

View full text Add to dashboard Cite

a b s t r a c tA prevailing theory regarding the evolution of language implicates a gestural stage prior to the emergence of speech. In support of a transition of human language from a gestural to a vocal system, articulation of the hands and the tongue are underpinned by overlapping left hemisphere dominant neural regions. Behavioral studies demonstrate that human adults perform sympathetic mouth actions in imitative synchrony with manual actions. Additionally, right-handedness for precision manual actions in children has been correlated with the typical development of language, while a lack of hand bias has been associated with psychopathology. It therefore stands to reason that sympathetic mouth actions during fine precision motor action of the hands may be lateralized. We employed a fine-grained behavioral coding paradigm to provide the first investigation of tongue protrusions in typically developing 4-year old children. Tongue protrusions were investigated across a range of cognitive tasks that required varying degrees of manual action: precision motor action, gross motor action and no motor actions. The rate of tongue protrusions was influenced by the motor requirements of the task and tongue protrusions were significantly right-biased for only precision manual motor action (p < .001). From an evolutionary perspective, tongue protrusions can drive new investigations regarding how an early human communication system transitioned from hand to mouth. From a developmental perspective, the present study may serve to reveal patterns of tongue protrusions during the motor development of typically developing children.

show abstract

Synchronization of speech and gesture: Evidence for interaction in action.

Cited by 45 publications

References 27 publications

Co-thought and co-speech gestures are generated by the same action generation process.

Co-thought and co-speech gestures are generated by the same action generation process.

Reduction in gesture during the production of repeated references

Slip of the tongue: Implications for evolution and language development

Contact Info

Product

Resources

About