Sound Pattern Matching for Automatic Prosodic Event Detection

Abstract

Prosody in speech is manifested by variations of loudness, exaggeration of pitch, and specific phonetic variations of prosodic segments. For example, in the stressed and unstressed syllables, there are differences in place or manner of articulation, vowels in unstressed syllables may have a more central articulation, and vowel reduction may occur when a vowel changes from a stressed to an unstressed position.In this paper, we characterize the sound patterns using phonological posteriors to capture the phonetic variations in a concise manner. The phonological posteriors quantify the posterior probabilities of the phonological classes given the input speech acoustics, and they are obtained using the deep neural network (DNN) computational method. Built on the assumption that there are unique sound patterns in different prosodic segments, we devise a sound pattern matching (SPM) method based on 1-nearest neighbour classifier. In this work, we focus on automatic detection of prosodic stress placed on words, called also emphasized words. We evaluate the SPM method on English and French data with emphasized words. The word emphasis detection works very well also on cross-lingual tests, that is using a French classifier on English data, and vice versa. Index Terms: Automatic prosodic event detection, word emphasis, phonological posteriors, nearest neighbour rule of classification.

On the impact of non-modal phonation on phonological features

“…We use our open-source phonological vocoding platform [16] to perform phonological analysis and synthesis. Briefly, the platform is based on cascaded speech analysis and synthesis that works internally with the phonological speech representation.…”

Acoustic and Visual Stimulus Parameters Underlying Sound Symbolic Crossmodal Correspondences

“…The Jaccard distance and simple matching coefficient (the latter is also known as the Rand similarity coefficient) are two types of distance measures that have been used to compare similarity and diversity, including patterns in DNA (Deagle et al, 2017), images (Devereux et al, 2013), and voices (Cernak et al, 2016). The spatial frequency FFT has been used to filter noisy or complex patterns in complex image sets (Petrou & Petrou, 2011).…”

Emphasis recreation for TTS using intonation atoms

“…Using an emphasis detection module combined with ASR-based automatic time alignment, it is possible to identify which word is emphasised in a sentence and its boundaries (we do not tackle this problem in this work; it can be solved using different methods, e.g. [11,12]). In our previous work [10], given parallel data including neutral and emphasised speech, by retrieving the parameters of our model for both cases, we showed that adding the most prominent atoms from an emphatic word in a neutral sentence consistently increased the perception of emphasis on the target word.…”

“…In the more general framework of translating emphasis in S2ST, some emphasis detection system (e.g. the recent work of Cernak and colleagues [11,12]) can be used to provide the machine translation additional information, which can further be transmitted to the TTS system. For this study, we restrict ourselves to the intra-lingual case, but due to the language independence of the intonation model used, it seems reasonable to assume that this method can work for any given language.…”