A well-known phenomenon of multimodal language is the synchronous coupling of prosodic contours in speech with salient kinematic changes in co-speech hand-gesture motions. Invariably, such coupling has been rendered by psychologists to require a dedicated neural-cognitive mechanism preplanning speech and gesture trajectories. Recently, in a continuous vocalization task, it was found that acoustic peaks unintentionally appear in vocalizations when gesture motions reach peaks in physical impetus, suggesting a biomechanical basis for gesture-speech synchrony (Pouw, Harrison, & Dixon, 2019). However, from this rudimentary study it is still difficult to draw strong conclusions about gesture-speech dynamics in (more) complex speech and the precise biomechanical nature of these effects. Here we assess how the timing of physical impetus of a gesture relates to its effect on acoustic parameters of mono-syllabic consonant-vowel (CV) vocalization(/pa/). Furthermore, we assess how chest-wall kinematics is affected by gesturing, and whether this modulates the effect of gestures on acoustics. In the current exploratory analysis, we analyze a subset (N = 4) of an already collected dataset (N = 36), which serves as the basis for a pre-registration of the confirmatory analyses yet to be completed. Here we provide exploratory evidence that gestures affect acoustics (amplitude envelope and F0) as well as chest-wall kinematics during mono-syllabic vocalizations. These effects are more extreme when a gesture’s peak impetus occurs closer to the center of the vowel vocalization event. If the current findings can be replicated in confirmatory fashion, there is a more compelling case to be made that gesture-speech physics is important facet of multimodal synchrony.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.