Postural and muscular effects of upper-limb movements on voicing

Pouw, Wim; Burchardt, Lara S.; Selen, Luc P. J.

doi:10.1101/2023.03.08.531710

Cited by 3 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We should further highlight that our hypothesized mechanism of vocal-motor interaction is on the level of biomechanics (kinetics) suggesting that mechanical loading on the thorax increases eggressive flow for vocalization (raising amplitude), but our analysis is on the level of movement (kinematics). Accordingly more work is needed on the level of biomechanics too 5,10,30,44 , evaluating how locomotion might affect the surrounding myofascial-skeletal structure around the respiratory system and thereby far-from-equilibrium subglottal pressures needed for vocalizing.…”

Section: Discussionmentioning

confidence: 99%

Cross-modal constraints in multimodal vocalizations in Siamang (Syndactylus symphalangus)

Pouw¹,

Kehy²,

Gamba³

et al. 2023

Preprint

View full text Add to dashboard Cite

Gibbons (Hylobatidae) sing loudly, reaching over a hundred Decibels - about the sound level of a rock concert. Qualitative observations report that, during song climaxes, individuals move in a coordinated way with their singing. We hypothesizethat vigorous thorax-loading movements such as brachiation induce physical constraints on the respiratory-vocal system. This coordination and possibly underlying biomechanics have never been studied, leaving a blind spot for a possible sharedvariance between vocal and locomotor repertoires that would indicate a co-evolution of two systems. Here, we recorded over a hundred stereotypical multimodal vocal calls from over 7 hours of singing in two captive siamangs (Syndactylus symphalangus), a type of gibbon that are the largest in size. These stereotypical calls coincided with a bodily display during solo singing, and were exclusively performed by juvenile individuals. We used computer vision methods (opencv and deeplabcut) to quantify upper body acceleration of these multimodal displays of two individuals; we found that body accelerationstatistically predicted the nearest peak in the amplitude envelope of the call. The results indicate that Siamang singing likely co-evolved with movement due to physical constraints of pectoral limb-respiratory-vocal interactions, similar to birds, bats, and rodents, as well as humans. This has important implications for the locomotor hypotheses of rhythm origins as our results suggest a biomechanical and basic coupling of vocal systems with the (loco)-motor system: in singing Siamangs, and perhaps humans too, as there are homologies to be drawn between how humans and Siamang move in gestural ways with their vocalizations.

show abstract

Section: Discussionmentioning

confidence: 99%

Cross-modal constraints in multimodal vocalizations in Siamang (Syndactylus symphalangus)

Pouw¹,

Kehy²,

Gamba³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Research into gesture-speech physics has conducted experiments and synthesized findings from neural processes, biomechanics and social interaction to develop a 'multilevel multimodal approach' to gesture and prosody (Pouw et al, 2021;cf. Pouw, de Jonge-Hoekstra, et al, 2020a;Pouw, Harrison, et al, 2020b, Pouw & Fuchs, 2022, Pouw et al, 2024. Of interest here is the 'gesture-speech biomechanics thesis', which stems from empirical evidence that 'when an upper-limb segment with a certain mass (or multiple segments with a certain combined mass) sufficiently accelerates or decelerates, it yields physical impulses on the musculoskeletal system, the cascading mechanical effects of which will constraining respiratory-vocal activity' (Pouw & Fuchs, 2022, p. 4).…”

Section: Gesture-speech Biomechanics and Vocal-entangled Gestures Ass...mentioning

confidence: 99%

“…The research into 'vocal-entangled gesture', on the other hand, reveals correlations at scales and speeds that are happening 'deeper' and/or at 'higher frequency' and which have their own vitality and animacy. Such correlations include those within the bio-anatomy of our respiratory-motor systems discovered experimentally with various muscle, posture, motion and audiosensing technologies (Pouw et al, 2024;Pouw & Fuchs, 2022). When compared to the neat partitioning of components or activities into audio-oral and visualgestural modes from where multimodal treatments of speech and gesture often set out (cf.…”

Section: Introductionmentioning

confidence: 99%

‘This you may NNNNNNEVER have heard before’: initial lengthening of accented negative items as vocal-entangled gestures

Harrison

2024

Lang. cogn.

View full text Add to dashboard Cite

Movement scientists have proposed to ground the relation between prosody and gesture in ‘vocal-entangled gestures’, defined as biomechanical linkages between upper limb movement and the respiratory–vocal system. Focusing on spoken language negation, this article identifies an acoustic profile with which gesture is plausibly entangled, specifically linking the articulatory behaviour of onset consonant lengthening with forelimb gesture preparation and facial deformation. This phenomenon was discovered in a video corpus of accented negative utterances from English-language televised dialogues. Eight target examples were selected and examined using visualization software to analyse the correspondence of gesture phase structures (preparation, stroke, holds) with the negation word’s acoustic signal (duration, pitch and intensity). The results show that as syllable–onset consonant lengthens (voiced alveolar /n/ = 300 ms on average) with pitch and intensity increasing (e.g. ‘NNNNNNEVER’), the speaker’s humerus is rotating with palm pronating/adducing while his or her face is distorting. Different facial distortions, furthermore, were found to be entangled with different post-onset phonetic profiles (e.g. vowel rounding). These findings illustrate whole-bodily dynamics and multiscalarity as key theoretical proposals within ecological and enactive approaches to language. Bringing multimodal and entangled treatments of utterances into conversation has important implications for gesture studies.

show abstract

“…Importantly, despite the gesture-speech physics theory clearly assuming a respiratory effect of beat-like upper-limb movements, no measurements have been taken to directly assess respiratory modulation. In this exploratory study, we thus analyze the data of participants producing voiceless expirations that were recorded along with and under the same experimental conditions as voicing [16,19]. We assess whether different types of arm movement affect expiration as compared to a passive no movement condition.…”

Section: Introductionmentioning

confidence: 99%

Arm movements increase acoustic markers of expiratory flow

Werner,

Selen,

Pouw

2024

Preprint

Self Cite

View full text Add to dashboard Cite

The gesture-speech physics theory suggests that there are biomechanical interactions of the voice with the whole body, driving speech to align fluctuations in loudness and F0 with upper-limb movement. This exploratory study offers a possible falsification of the gesture-speech physics theory, which would predict effects of upper-limb movement on voice as well as respiration. We therefore investigate co-movement expiration. Seventeen participants were asked to produce a continuous exhalation for several seconds. After 3s, they execute one of five within-subject movement conditions with their arm with and without a wrist weight (no movement, elbow flexion, elbow extension, internal arm rotation, external arm rotation). We analyzed the smoothed amplitude envelope of the acoustic signal in relation to arm movement. Compared to no movement, all four movements lead to higher positive peaks in the amplitude peaks, while weight did not influence the amplitude. We also found that across movement conditions, positive amplitude peaks are structurally timed relative to peaks in kine-matics (speed, acceleration). We conclude that the reason why upper-limb movements affect voice loudness is still best understood through gesture-speech physics theory, where upper-limb movements affect the voice directly by modulating sub-glottal pressures. Multimodal prosody is therefore partly literally embodied.

show abstract

Postural and muscular effects of upper-limb movements on voicing

Cited by 3 publications

References 22 publications

Cross-modal constraints in multimodal vocalizations in Siamang (Syndactylus symphalangus)

Cross-modal constraints in multimodal vocalizations in Siamang (Syndactylus symphalangus)

‘This you may NNNNNNEVER have heard before’: initial lengthening of accented negative items as vocal-entangled gestures

Arm movements increase acoustic markers of expiratory flow

Contact Info

Product

Resources

About