A Covariance-Tying Technique for HMM-Based Speech Synthesis

Abstract

SUMMARYA technique for reducing the footprints of HMM-based speech synthesis systems by tying all covariance matrices of state distributions is described. HMM-based speech synthesis systems usually leave smaller footprints than unit-selection synthesis systems because they store statistics rather than speech waveforms. However, further reduction is essential to put them on embedded devices, which have limited memory. In accordance with the empirical knowledge that covariance matrices have a smaller impact on the quality of synthesized speech than mean vectors, we propose a technique for clustering mean vectors while tying all covariance matrices. Subjective listening test results showed that the proposed technique can shrink the footprints of an HMM-based speech synthesis system while retaining the quality of the synthesized speech.

An optimization algorithm of independent mean and variance parameter tying structures for HMM-based speech synthesis

“…However, it may not be always appropriate that mean and variance parameters have the same tying structure. As an example, we confirmed the effectiveness of a technique for context clustering mean vectors while tying all variance matrices [4]. In this technique, the synthesized speech can be expected to improve by constructing different tying structures for both mean and variance parameters.…”

“…In[4], variance parameters are tied to one in all states of HMMs. In this paper, we assume that the technique with the enough big weight of the MDL criterion in the proposed technique is the conventional one.…”

Context-dependent acoustic modeling based on hidden maximum entropy model for statistical parametric speech synthesis

“…In this way, the final HMEM preserves the first empirical moments of multiple decision trees, and the second moments of just one decision tree. This assumption is a result of the fact that first-order moments seem to be more important than second-order moments [32,47]. The discussion of current section shows that the ML estimates of parameters defined for…”

“…Due to the prime importance of mean parameters in HMM-based speech synthesis [47], we investigate the difference between mean values predicted by two systems. Figure 3A shows a three-dimensional contextual factor space (c 1 -c 2 -c 3 ) which is clustered by an additive structure.…”

Speech Synthesis Based on Hidden Markov Models

“…It was demonstrated that HMM-based speech synthesis systems whose footprints were about 100 KBytes could synthesize intelligible speech by using vector quantization, fixed-point numbers instead of floating-point numbers, and pruned decision trees. In addition, several techniques that suit embedded devices have been proposed, e.g., memory-efficient, low-delay speech parameter generation algorithms [87], [88] and tying model parameters [89].…”