Dynamic Imagery in Speech and Gesture

McNeill, David; McCullough, Karl-Erik; Quek, Francis; Duncan, Susan; Bryll, Robert; Ma, Xinfeng; Ansari, Rashid

doi:10.1007/978-94-017-2367-1_3

Cited by 7 publications

(7 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, multiple sequenced beat gestures may move with the rhythm of speech (operating on the scale of milliseconds; Leonard & Cummins, 2010); gesture sequences may reflect syntactic conventions that are present on the verbal sentential level (seconds; Kita & Ozyurek, 2003). Furthermore, some gestures seem to retain similarity in their kinematic profiles as they recur at several times in a discourse, providing anchors for discourse cohesion (minutes; McNeill et al, 2002). As such, if gesture and speech differ in their coupling under DAF vs. NO DAF, such changes in correlation between temporal coupling of gesture and speech might be present on the multiple time scales (e.g., phonemes to syllables to sentences).…”

Section: Current Paradigmmentioning

confidence: 99%

Entrainment and Modulation of Gesture–Speech Synchrony Under Delayed Auditory Feedback

Pouw

Dixon

2019

Cognitive Science

View full text Add to dashboard Cite

Gesture–speech synchrony re‐stabilizes when hand movement or speech is disrupted by a delayed feedback manipulation, suggesting strong bidirectional coupling between gesture and speech. Yet it has also been argued from case studies in perceptual–motor pathology that hand gestures are a special kind of action that does not require closed‐loop re‐afferent feedback to maintain synchrony with speech. In the current pre‐registered within‐subject study, we used motion tracking to conceptually replicate McNeill's ( 1992 ) classic study on gesture–speech synchrony under normal and 150 ms delayed auditory feedback of speech conditions ( NO DAF vs. DAF ). Consistent with, and extending McNeill's original results, we obtain evidence that (a) gesture‐speech synchrony is more stable under DAF versus NO DAF (i.e., increased coupling effect), (b) that gesture and speech variably entrain to the external auditory delay as indicated by a consistent shift in gesture‐speech synchrony offsets (i.e., entrainment effect), and (c) that the coupling effect and the entrainment effect are co‐dependent. We suggest, therefore, that gesture–speech synchrony provides a way for the cognitive system to stabilize rhythmic activity under interfering conditions.

show abstract

Section: Current Paradigmmentioning

confidence: 99%

Entrainment and Modulation of Gesture–Speech Synchrony Under Delayed Auditory Feedback

Pouw

Dixon

2019

Cognitive Science

View full text Add to dashboard Cite

show abstract

“…In studies of speech production, McNeill et al (2002) have argued that speech and movement gestures arise from a shared semantic source. Head shakes or body sway are examples of movements that, although not actually producing sound, still can serve a communicative purpose of their own.…”

Section: Ody Movement Is An Important Non-verbalmentioning

confidence: 99%

Visual Perception of Expressiveness in Musicians' Body Movements

2007

View full text Add to dashboard Cite

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.. University of California Press is collaborating with JSTOR to digitize, preserve and extend access to Music Perception: An Interdisciplinary Journal. MUSICIANS OFTEN MAKE GESTURES and move their bodies expressing a musical intention. In order to explore to what extent emotional intentions can be conveyed through musicians' movements, participants watched and rated silent video clips of musicians performing the emotional intentions Happy, Sad, Angry, and Fearful. In the first experiment participants rated emotional expression and movement character of marimba performances. The results showed that the intentions Happiness, Sadness, and Anger were well communicated, whereas Fear was not. Showing selected parts of the player only slightly influenced the identification of the intended emotion. In the second experiment participants rated the same emotional intentions and movement character for performances on bassoon and soprano saxophone. The ratings from the second experiment confirmed that Fear was not communicated whereas Happiness, Sadness, and Anger were recognized. The rated movement cues were similar in the two experiments and were analogous to their audio counterpart in music performance.

show abstract

“…Furthermore, some gestures seem to retain similarity in their kinematic profiles as they recur at several times in a discourse, providing anchors for discourse cohesion (minutes;McNeill et al, 2002). As such, if gesture and speech differ in their coupling under DAF vs.…”

Section: Current Paradigmmentioning

confidence: 99%

Entrainment and Modulation of Gesture-Speech Synchrony under Delayed Auditory Feedback

Pouw¹,

Dixon²

2018

Preprint

View full text Add to dashboard Cite

Gesture-speech synchrony re-stabilizes when hand movement or speech is disrupted by a delayed feedback manipulation, suggesting strong bidirectional coupling between gesture and speech. Yet, it has also been argued from case studies in perceptual-motor pathology, that hand gestures are a special kind of action that do not require closed-loop re-afferent feedback to maintain synchrony with speech. In the current pre-registered within-subject study, we used motion tracking to conceptually replicate McNeill’s (1992) classic study on gesture-speech synchrony under normal and 150ms delayed auditory feedback of speech conditions (NO DAF vs. DAF). Consistent with, and extending McNeill’s original results, we obtain strong evidence that: a) gesture-speech synchrony is more stable under DAF versus NO DAF (i.e., increased coupling effect), b) that gesture and speech variably entrain to the external auditory delay as indicated by a consistent shift in gesture-speech synchrony offsets (i.e., entrainment effect), and c) that the coupling effect and the entrainment effect are co-dependent. We suggest, therefore, that gesture-speech synchrony provides a way for the cognitive system to stabilize rhythmic activity under interfering conditions.

show abstract

Dynamic Imagery in Speech and Gesture

Cited by 7 publications

References 6 publications

Entrainment and Modulation of Gesture–Speech Synchrony Under Delayed Auditory Feedback

Entrainment and Modulation of Gesture–Speech Synchrony Under Delayed Auditory Feedback

Visual Perception of Expressiveness in Musicians' Body Movements

Entrainment and Modulation of Gesture-Speech Synchrony under Delayed Auditory Feedback

Contact Info

Product

Resources

About