Strategies for developing a Conversational Speech Dataset for Text-To-Speech Synthesis

Abstract

There have been many efforts to improve the quality of speech synthesis systems in conversational AI. Although state-of-theart systems are capable of producing natural-sounding speech, the generated speech often lacks prosodic variation and is not always suited to the task. In this paper, we examine dialogue data collection methods to use as training data for our acoustic models. We collect speech using three different setups: (1) Random read-aloud sentences; (2) Performed dialogues; (3) Semi-Spontaneous dialogues. We analyze prosodic and textual properties of the data collected in these setups and make some recommendations to collect data for speech synthesis in conversational AI settings.

Prosody-Controllable Spontaneous TTS with Neural HMMS

“…Moreover, the prosody is generated solely from text, which allows no control over the generated style [5]. Concurrently, spontaneous speech is increasingly used in TTS [6,7]. Spontaneous speech data is challenging to model, due to disfluencies and large variability [8]; offers high ecological validity for evermore commonplace conversational AI systems, and the varied prosody offered by spontaneous data positively impacts factors such as word recall and attention [9].…”