Dimensional paralinguistic information control based on multiple-regression HSMM for spontaneous dialogue speech synthesis with robust parameter estimation

Nagata, T.; Mori, Hiroki; Nose, Takeru

doi:10.1016/j.specom.2017.01.002

Cited by 13 publications

(10 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Controllability of paralinguistic information with the proposed deep neural network (DNN) was investigated by comparison with the method based on the MAP-MRHSMM [4]. Figure 2 shows the change in the distribution of averaged fundamental frequency of synthesized utterances, with given 2-dimensional values of paralinguistic information.…”

Section: Analysis Of Synthesized Speechmentioning

confidence: 99%

“…Previously, we studied a dialogue speech synthesis based on multiple-regression hidden semi-Markov model (MRHSMM) [4]. Although the MRHSMM enabled to control paralinguistic information in the form of dimensions such as pleasantunpleasant, aroused-sleepy, etc., synthesized speech tended to have extreme parameters due to badly estimated regression matrices.We have shown that MAP estimation of regression matrices was effective to reduce the overfitting problem [4]. How-ever, the problem still remains for certain combinations of given input as paralinguistic information.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Effects of Dimensional Input on Paralinguistic Information Perceived from Synthesized Dialogue Speech with Neural Network

Yokoyama¹,

Nagata

Mori

2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

A novel method of controlling paralinguistic information in neural network-based dialogue speech synthesis is proposed. Controlling paralinguistic information was achieved by feeding emotion dimensions in continuous values into the input layer of the neural networks. Compared to the method using the multiple regression HMM, the naturalness of synthesized speech was improved. The controllability of paralinguistic information was evaluated by examining the shift of the distribution of synthesized parameters. A subjective evaluation test revealed that the correlation between given and perceived paralinguistic information was moderate, though less apparent compared to the multiple regression HMM-based method.

show abstract

Section: Analysis Of Synthesized Speechmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Effects of Dimensional Input on Paralinguistic Information Perceived from Synthesized Dialogue Speech with Neural Network

Yokoyama¹,

Nagata

Mori

2018

Interspeech 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…While there has been some past work on building speech synthesisers from spontaneous speech audio [3,4,5,6], this was restricted to small, hand-annotated corpora and statistical parametric speech synthesisers. Bigger corpora can usually be sourced from found speech recordings, but the output quality has been disappointing, e.g., [7].…”

Section: Introductionmentioning

confidence: 99%

Spontaneous Conversational Speech Synthesis from Found Data

et al. 2019

View full text Add to dashboard Cite

Synthesising spontaneous speech is a difficult task due to disfluencies, high variability and syntactic conventions different from those of written language. Using found data, as opposed to labrecorded conversations, for speech synthesis adds to these challenges because of overlapping speech and the lack of control over recording conditions. In this paper we address these challenges by using a speaker-dependent CNN-LSTM breath detector to separate continuous recordings into utterances, which we here apply to extract nine hours of clean single-speaker breath groups from a conversational podcast. The resulting corpus is transcribed automatically (both lexical items and filler tokens) and used to build several voices on a Tacotron 2 architecture. Listening tests show: i) pronunciation accuracy improved with phonetic input and transfer learning; ii) it is possible to create a more fluent conversational voice by training on data without filled pauses; and iii) the presence of filled pauses improved perceived speaker authenticity. Another listening test showed the found podcast voice to be more appropriate for prompts from both public speeches and casual conversations, compared to synthesis from found read speech and from a manually transcribed lab-recorded spontaneous conversation.

show abstract

“…The most commonly used evaluation protocols for TTS are listening tests with respect to quality, naturalness, intelligibility, similarity and expressiveness [7,8,9,10]. Applicationdependent measures are also used, such as those for audiobook reading [11] and spoken dialogue systems or human-robot interaction [12,13,14,15,16,17,18].…”

Section: Introductionmentioning

confidence: 99%

“…For example, interpretation about what is and what is not "CRITICISM" can be inconsistent among participants and researchers. This problem may not sound serious when a limited number of categories [4,5] or low-dimensional representation [10] are of interest. However, it becomes severe when using larger number of intentions for evaluation because it will be harder to precisely comprehend their difference.…”

Section: Introductionmentioning

confidence: 99%

Evaluating Intention Communication by TTS Using Explicit Definitions of Illocutionary Act Performance

Hojo

Miyazaki²

2019

Interspeech 2019

View full text Add to dashboard Cite

Text-to-speech (TTS) synthesis systems have been evaluated with respect to attributes such as quality, naturalness and intelligibility. However, an evaluation protocol with respect to communication of intentions has not yet been established. Evaluating this sometimes produce unreliable results because participants can misinterpret definitions of intentions. This misinterpretation is caused by the colloquial and implicit description of intentions. To address this problem, this work explicitly defines each intention following theoretical definitions, "felicity conditions", in speech-act theory. We define the communication of each intention with one to four necessary and sufficient conditions to be satisfied. In listening tests, participants rated whether each condition was satisfied or not. We compared the proposed protocol with the conventional baseline using four different voice conditions; neutral TTS, conversational TTS w/ and w/o intention inputs, and recorded speech. The experimental results with 10 participants showed that the proposed protocol produced smaller within-group variation and larger betweengroup variation. These results indicate that the proposed protocol can be used to evaluate intention communication with higher inter-rater reliability and sensitivity.

show abstract

Dimensional paralinguistic information control based on multiple-regression HSMM for spontaneous dialogue speech synthesis with robust parameter estimation

Cited by 13 publications

References 14 publications

Effects of Dimensional Input on Paralinguistic Information Perceived from Synthesized Dialogue Speech with Neural Network

Effects of Dimensional Input on Paralinguistic Information Perceived from Synthesized Dialogue Speech with Neural Network

Spontaneous Conversational Speech Synthesis from Found Data

Evaluating Intention Communication by TTS Using Explicit Definitions of Illocutionary Act Performance

Contact Info

Product

Resources

About