Pushing the envelope: Evaluating speech rhythm with different envelope extraction techniques

Abstract

The amplitude of the speech signal varies over time, and the speech envelope is an attempt to characterise this variation in the form of an acoustic feature. Although tacitly assumed, the similarity between the speech envelope-derived time series and that of phonetic objects (e.g., vowels) remains empirically unestablished. The current paper, therefore, evaluates several speech envelope extraction techniques, such as the Hilbert transform, by comparing different acoustic landmarks (e.g., peaks in the speech envelope) with manual phonetic annotation in a naturalistic and diverse dataset. Joint speech tasks are also introduced to determine which acoustic landmarks are most closely coordinated when voices are aligned. Finally, the acoustic landmarks are evaluated as predictors for the temporal characterisation of speaking style using classification tasks. The landmark that performed most closely to annotated vowel onsets was peaks in the first derivative of a human audition-informed envelope, consistent with converging evidence from neural and behavioural data. However, differences also emerged based on language and speaking style. Overall, the results show that both the choice of speech envelope extraction technique and the form of speech under study affect how sensitive an engineered feature is at capturing aspects of speech rhythm, such as the timing of vowels.

Neural decoding of the speech envelope: Effects of intelligibility and spectral degradation

“…Audio was filtered to conform to the international long-term average speech spectrum (Byrne et al, 1994; Figure 1, Panel A). Long pauses ( > 500 ms) were manually restricted to 500 ms. Syllabic timing was estimated by windowing the speech recordings into 2 s chunks and automatically detecting vowel onset-like acoustic landmarks (MacIntyre et al, 2022). The grand average inter-vowel onset interval was, for English, 203.47 ms (SD 47.11 ms) and for Dutch, 202.68 ms (SD 38.96 ms).…”

“…Long pauses (> 500 ms) were manually removed. Syllabic timing was estimated by windowing the speech recordings into 2 s chunks and automatically detecting vowel onset-like acoustic landmarks [84]. The grand average inter-vowel onset interval was, for English, 203.47 ms (SD 47.11 ms) and for Dutch, 202.68 ms (SD 38.96 ms).…”

There is a power law of joint communicative effort and it reflects communicative work

“…To operationalize speech effort, we extracted the speech amplitude envelope from the audio recordings of the interactions for each participant (Pouw & Trujillo, 2021), with a smoothing Hanning filter of 5Hz and a resampling rate of 100Hz. We then counted the number of envelope peaks per trial as a proxy for the number of syllables (see e.g., MacIntyre et al, 2022), with a peak height threshold of 0.37 (set by M-(2*SD) envelope height over all participants). Gesture Effort.…”

Gesture–speech coupling in persons with aphasia: A kinematic-acoustic analysis.

“…For the R code to extract the amplitude envelope from speech please see Pouw and Trujillo (2019). For our analysis, we also take the first derivative of the amplitude envelope (the change in amplitude envelope) to measure the peak attack phase of the signal (i.e., sudden rises of the amplitude envelope; see e.g., MacIntyre, Cai, & Scott, 2022).…”