Waveform-based speech synthesis approach with a formant frequency modification

Abstract

This paper proposes a new approach to speech synthesis based on waveform segments. One novel point of this approach is its new formant frequency modification algorithm which makes it possible to flexibly change formant frequency and so reproduce the desired speech quality. The algorithm characterizes speech formants not only by formant frequencies and formant bandwidths, but also by spectral intensities of formant frequencies. The desirable formant structure, which is specified by the parameters, is obtained by iteratively modifying the formant bandwidths. Using the specified formant structure, the speech signal is synthesized by FFT. Evaluation by the acoustic distance measure, and by listening tests, confiis the good performance of the approach. IM'RODUCTIONIt has been reported that a text-to-speech system based on time-domain prosodic modification can synthesize high quality speech[ 11. We developed a text-to-speech system using a similar approach, and an earlier report confiied the good performance of our system[2]. The unique ideas and advantages of our system are as follows:(1)phoneme segment units are selected from a large database (45.000 phoneme segments) which consists of not only word utterances, but also sentence utterances. This makes it possible to locate more appropriate phoneme segments characterized by phoneme environment, sentence structure, and so on.(2)Optimum phoneme segments are selected considering various factors, such as phoneme environment, phoneme duration, pitch frequency contour, and speech power. This makes it possible to synthesize more natural speech.In this paper, we propose a new formant fiequency modification algorithm, and introduce the algorithm into a waveform-based speech synthesis system. There are two aims. One is to overcome the limitation of phoneme variety caused by the size of the speech database. Through system performance evaluation, we found that our original range of phonemes was insufficient, especially for some rare phoneme environments. In these cases. formant frequency modification is necessary. The other aim is to synthesize speech with various qualities.such as male speech, female speech, childish speech, husky speech, and so on [3][4][5]. Because constructing a large database is expensive and time consuming, to control speech quality by formant frequency modification is much preferable to constructing a database that covers a wide variety of speakers.The novel points of the proposed algorithm are (1) introducing spectral intensity as a parameter to specify the formant structure. and (2)iteratively modifying speech to produce the specified formant structure. Section 2 introduces the new formant frequency modification algorithm. In section 3, to c o n f i i the proposed algorithm performance, formant structures are iteratively modified so as to reproduce the desired formant structure. Section 4 applies the proposed algorithm to a waveform-based speech synthesis system. A NEW FORMANT FREQUENCY MODJFlCATlONAlJ." Outline of the algorithmIn the AR model of speech,...

Spectral Enhancement of Whispered Speech Based on Probability Mass Function

“…Although this kind of formant modification can be classified as direct modification, bundling the formant frequencies and weighting them based on their probabilities causes the pole interaction problem [13], [14] to be avoided. The next step is bandwidth improvement whilst preserving the new de-noised phases where the pole interaction effect is taken into account.…”

A flexible spectral modification method based on temporal decomposition and Gaussian mixture model

“…Therefore, using formant frequency as a parameter can control other features that are directly connected to the speech production process. Conventional spectral modification methods, such as [7][8][9], often control formants to modify the speech spectrum. However, these methods are limited by their inability to independently control important formant characteristics such as amplitude and bandwidth, or to control the spectral shape.…”

“…In addition, some methods mentioned above [7,8] only mention how to modify the speech spectrum in a frame, and they [7][8][9] rarely deal with constraints between frames after modification. This limitation may cause a discontinuity problem between adjacent frames.…”

“…They can be classified into two popular approaches: linear prediction (LP)-based methods [7,8] and frequency warping methods [9]. LP-based methods are often affected by the pole interaction problem suffered by pole modification techniques.…”

“…LP-based methods are often affected by the pole interaction problem suffered by pole modification techniques. An iterative algorithm for overcoming pole interaction during formant modification was developed by Mizuno et al [7] This method produces spectral envelopes with desired formant amplitudes at the formant frequencies. However, the amplitude and bandwidth of each formant cannot be independently modified, since each formant's bandwidth is dependent on the magnitude of the corresponding pole.…”