Integrated Expression Prediction and Speech Synthesis From Text

Chen, Langzhou; Gales, Mark J. F.; Braunschweiler, Norbert; Knill, Kate

doi:10.1109/jstsp.2013.2294938

Cited by 13 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [3], a multiple linear regression model is presented to predict the most appropriate hidden Markov model (HMM) parametric voice style from a created set, and has also been implemented and evaluated including statistical models such as Gaussian mixture models (GMM) [2]. Moreover, the prediction of expressions from text and the synthesis of a particular expression have been integrated together [14]. In addition, prosody has been structured as a multi-level hierarchy for emotional speech synthesis [5], and its correlation with both hierarchical information structure and discourse has also been analyzed for speech synthesis purposes [15,16,17].…”

Section: Related Workmentioning

confidence: 99%

Paragraph Prosodic Patterns to Enhance Text-to-Speech Naturalness

Peiró-Lilja¹,

Farrús²

2018

Speech Prosody 2018

View full text Add to dashboard Cite

Speech synthesis has reached a reasonable high quality in recent years. However, there is still room for improvement in terms of naturalness and expressiveness when dealing with large multisentential discourse, since most text-to-speech synthesizers do not fully take into account the prosodic differences that have been observed in discourse units such as paragraphs. This work presents an implementation of paragraph-based prosodic patterns into the open-source MARYTTS platform, enriching its prosody output by means of intra-and inter-paragraph prosodic features. The set of characteristics include pitch decay, pitch range and speech rate variation (as intra-paragraph features), as well as paragraph break pauses and speech rate variation (as inter-paragraph features), previously analyzed in a large set of TED Talks and read-speech sections of the Spoken Wikipedia Corpus. The perception tests, performed both in English and German parametric voices, suggest that paragraph-based features should be further studied and taken into account on future implementations to synthesize large discourse speech.

show abstract

Section: Related Workmentioning

confidence: 99%

Paragraph Prosodic Patterns to Enhance Text-to-Speech Naturalness

Peiró-Lilja¹,

Farrús²

2018

Speech Prosody 2018

View full text Add to dashboard Cite

show abstract

“…Also, in order to account for the practically infinite amount of expressive styles, automatized techniques must be applied to deal with data. The aforementioned problem of predicting expressiveness from text is addressed in for instance Chen et al (2014); Jauk et al (2016); Jauk and Bonafonte (2016a); Lorenzo Trueba (2016). The methods proposed in these works can also be used for data clustering in order to gain training data, as done for example by Watts (2012)).…”

Section: Expressive Speech Synthesismentioning

confidence: 99%

Unsupervised Learning for Expressive Speech Synthesis

Jauk¹

2018

IberSPEECH 2018

View full text Add to dashboard Cite

Nowadays, especially with the upswing of neural networks, speech synthesis is almost totally data driven. The goal of this thesis is to provide methods for automatic and unsupervised learning from data for expressive speech synthesis. In comparison to "ordinary" synthesis systems, it is more difficult to find reliable expressive training data, despite huge availability on sources like Internet. The main difficulty consists in the highly speaker-and situation-dependent nature of expressiveness, causing many and acoustically substantial variations. The consequences are, first, it is very difficult to define labels which reliably identify expressive speech with all nuances. The typical definition of 6 basic emotions, or alike, is a simplification which will have inexcusable consequences dealing with data outside the lab. Second, even if a label set is defined, apart of the enormous manual effort, it is difficult to gain sufficient training data for the models respecting all the nuances and variations.

show abstract

“…Emotional Text-To-Speech (TTS) is a challenging but important part in speech synthesis since rendering emotion makes speech sound more natural [1]. It permits to convey essential nonlinguistic information that can be extracted from text in addition to the commonly modelled linguistic aspects, such as syllable stress and punctuation.…”

Section: Introductionmentioning

confidence: 99%

“…Without this mapping the speech synthesis system cannot determine the appropriate expression cluster for synthesizing a given sentence. In [1], the authors propose a method for expression prediction from text and speech, in which both the expression predictor and speech synthesizer share the same training data. This method permits to model intra-speaker and inter-speaker variabilities that influence expression prediction and represent a higher number of expressions than the typical limited set of emotions of text predictor methods.…”

Section: Introductionmentioning

confidence: 99%

“…We use the emotion labels of the sentiment analysis to automatically detect the clusters and utterances that represent each emotion. This approach has a limitation in the number of expression categories that can be detected in the audiobook compared with [1], because the latter uses a continuous expressive space which reduces the problems of labelling emotions and can synthesise speech with more detailed expressions. However, our approach has the advantage that it provides a higher degree of flexibility for manual control of the emotions of the synthetic voice and verification of emotion labels for supervised training of the classifiers.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Prediction of Emotions from Text using Sentiment Analysis for Expressive Speech Synthesis

Vanmassenhove¹,

Cabral²,

Haider³

2016

9th ISCA Workshop on Speech Synthesis Workshop (SSW 9)

View full text Add to dashboard Cite

The generation of expressive speech is a great challenge for text-to-speech synthesis in audiobooks. One of the most important factors is the variation in speech emotion or voice style. In this work, we developed a method to predict the emotion from a sentence so that we can convey it through the synthetic voice. It consists of combining a standard emotion-lexicon based technique with the polarity-scores (positive/negative polarity) provided by a less fine-grained sentiment analysis tool, in order to get more accurate emotion-labels. The primary goal of this emotion prediction tool was to select the type of voice (one of the emotions or neutral) given the input sentence to a stateof-the-art HMM-based Text-to-Speech (TTS) system. In addition, we also combined the emotion prediction from text with a speech clustering method to select the utterances with emotion during the process of building the emotional corpus for the speech synthesizer. Speech clustering is a popular approach to divide the speech data into subsets associated with different voice styles. The challenge here is to determine the clusters that map out the basic emotions from an audiobook corpus that contains high variety of speaking styles, in a way that minimizes the need for human annotation. The evaluation of emotion classification from text showed that, in general, our system can obtain accuracy results close to that of human annotators. Results also indicate that this technique is useful in the selection of utterances with emotion for building expressive synthetic voices.

show abstract

Integrated Expression Prediction and Speech Synthesis From Text

Cited by 13 publications

References 26 publications

Paragraph Prosodic Patterns to Enhance Text-to-Speech Naturalness

Paragraph Prosodic Patterns to Enhance Text-to-Speech Naturalness

Unsupervised Learning for Expressive Speech Synthesis

Prediction of Emotions from Text using Sentiment Analysis for Expressive Speech Synthesis

Contact Info

Product

Resources

About