Speech Processing and Prosody

Abstract

The prosody of the speech signal conveys information over the linguistic content of the message: prosody structures the utterance, and also brings information on speaker's attitude and speaker's emotion. Duration of sounds, energy and fundamental frequency are the prosodic features. However their automatic computation and usage are not obvious. Sound duration features are usually extracted from speech recognition results or from a force speech-text alignment. Although the resulting segmentation is usually acceptable on clean native speech data, performance degrades on noisy or not non-native speech. Many algorithms have been developed for computing the fundamental frequency, they lead to rather good performance on clean speech, but again, performance degrades in noisy conditions. However, in some applications, as for example in computer assisted language learning, the relevance of the prosodic features is critical; indeed, the quality of the diagnostic on the learner's pronunciation will heavily depend on the precision and reliability of the estimated prosodic parameters. The paper considers the computation of prosodic features, shows the limitations of automatic approaches, and discusses the problem of computing confidence measures on such features. Then the paper discusses the role of prosodic features and how they can be handled for automatic processing in some tasks such as the detection of discourse particles, the characterization of emotions, the classification of sentence modalities, as well as in computer assisted language learning and in expressive speech synthesis.

Speech Translation and the End-to-End Promise: Taking Stock of Where We Are

“…In particular, the MT model is unaware of prosody which structures and disambiguates the utterances, thus playing a role similar to punctuation in written texts; and provides ways to emphasize words or parts of the messages that the speaker think are important. Prosody also conveys information on the speaker's attitude and emotional state (Jouvet, 2019).…”

The Prosody of Directive Speech Acts: Pragmastylistic Aspect (On the Material of the Modern German Language)

“…Linguists working in the field of theoretical and applied Phonetics note that having the largest potential of speech alteration to achieve positive perlocution in the process of communication, prosody plays a key role in forming semantic and pragmatic value of utterances (Gerazov et al, 2018;Golovchanskaya, 2018;Gibbon & Selting, 1983;Grigoriev, 1997;Eremeev, 2000;Imo & Lanwer, 2019;Jouvet, 2019;Karandeeva, 2013;Khrakovskiy & Volodin, 1989;Kodzasov & Krivnova, 2001;Lentsa & Solovyeva, 1989;Marić, 2018;Meinhold & Stock, 1982;Müller, 1999;Potapova & Potapov, 2006;Selting, 1995;Chesnokova, 2015).…”