Gesture–vocal coupling in Karnatak music performance: A neuro–bodily distributed aesthetic entanglement

Abstract

In many musical styles vocalists manually gesture while they sing. Coupling between gesture kinematics and vocalization has been examined in speech contexts, but it is an open question how these couple in music-making. We examine this in a corpus of South Indian, Karnatak vocal music which includes motion-capture data. Through peak magnitude analysis (linear mixed regression) and continuous time-series analyses (GAM modeling), we assessed whether vocal trajectories around peaks in vertical velocity, speed or acceleration were coupling with changes in vocal acoustics (namely, F0 and amplitude). Kinematic coupling was stronger for F0 change versus amplitude, pointing to F0's musical significance. Acceleration was the most predictive for F0 change and had the most reliable magnitude coupling, showing a 1/3 power relation. That acceleration rather than other kinematics is maximally predictive for vocalization is interesting because acceleration entails force-transfers onto the body. As a theoretical contribution, we argue that gesturing in musical contexts should be understood in relation to the physical connections between gesturing and vocal production that are brought in harmony with the vocalists' (enculturated) performance goals. Gesture-vocal coupling should therefore be viewed as a neuro-bodily distributed aesthetic entanglement.

Gesture–speech coupling in persons with aphasia: A kinematic-acoustic analysis.

“…As can be seen from the graphical results reported in Figure 2, untransformed values for peak ΔENV did not relate linearly with peak acceleration, and we also did not find statistically reliable results for analyses with these variables. However, similar to previous research (Pearson & Pouw, 2022), these variables showed a power relationship, as indicated by linear trends emerging after log–log plotting of the data. We performed a linear mixed regression with the log-transformed peak ΔENV (log peak ΔENV) as a dependent variable and participants as a random intercept.…”

“…The biomechanical theory is to be differentiated from the aforementioned views sketched, in that it suggests that gesture and speech are coupled in a rudimentary way that is not completely governed by a dedicated linguistic system. However, while gesture–speech coupling may emerge from biomechanics, it is of course regulated by the prosodic regimes of the particular language that is spoken, or the vocal music that is sung (Pearson & Pouw, 2022).…”

Leg movements affect speech intensity

“…The authors also showed that the acoustic events occurred in the vicinity of the acceleration peak. Although the original work was based on controlled speech units ( 6 , 7 ), the relationship has been recently confirmed with natural speech ( 13 ) and partially in Indian vocal music ( 42 ). In the current study, we extended this work to cyclical movements of the arms or the legs with low physical effort during spontaneous narration.…”

Postural and muscular effects of upper-limb movements on voicing

“…Yet, when humans speak or sing it is common to move the upper limbs expressively at the same time called gesture 11,20 . Such upper limb movements will recruit a whole range of upper body muscles, including those involved in maintaining posture 4 .…”