Exemplar-based Speech Waveform Generation

Abstract

This paper presents a simple but effective method for generating speech waveforms by selecting small units of stored speech to match a low-dimensional target representation. The method is designed as a drop-in replacement for the vocoder in a deep neural network-based text-to-speech system. Most previous work on hybrid unit selection waveform generation relies on phonetic annotation for determining unit boundaries, or for specifying target cost, or for candidate preselection. In contrast, our waveform generator requires no phonetic information, annotation, or alignment. Unit boundaries are determined by epochs, and spectral analysis provides representations which are compared directly with target features at runtime. As in unit selection, we minimise a combination of target cost and join cost, but find that greedy left-to-right nearest-neighbour search gives similar results to dynamic programming. The method is fast and can generate the waveform incrementally. We use publicly available data and provide a permissively-licensed open source toolkit for reproducing our results.

Examplar-Based Speechwaveform Generation for Text-To-Speech

“…We found that this simple search was sufficient as units are too short to deviate from the target sequence in the course of a single unit [8].…”

“…From these indices a sequence of higher dimension acoustic features is created and used for waveform reconstruction. In this section we will summarise the waveform generation method proposed in [8] that forms the basis of the hybrid TTS framework proposed in this paper.…”

“…As in the case of other examplar-based approaches, the proposed method selects a sequence of speech segments under two types of constraint: that each unit should be acoustically close to its target (divergence is penalised with a target cost), and that the end of each unit in the sequence should be acoustically similar to the start of the following unit, so that they can be joined without audible artefacts (implemented with a join cost). As mentioned in [8], the target and join components of the combined cost can be regarded as measures of fidelity and fluency respectively, the first scoring how faithfully the desired message is encoded and the second, how fluently it is rendered. In the following subsections we detail how the database of natural speech waveform units is created and how to generate new waveforms from this database.…”

“…We then extract spectral features characterising the signal around each of these pitchmarks, through a pitch-synchronous analysis. Following [8], the term frame from now on denotes a pitchmark-centred acoustic feature vector.…”

“…To resolve these issues the current work proposes a hybrid TTS that uses an examplar-based waveform generation method based on smaller units which are determined without phonetic annotation. This waveform generation system was first proposed in [8]; in this paper we integrate it with a TTS acoustic model and present its halfphone variant that was used in [9]. Similar small unit systems have been proposed before, where units are determined without phonetic Links to audio samples and code for recreating the systems described here can be found at https://github.com/CSTR-Edinburgh/ snickery.…”