A flexible spectral modification method based on temporal decomposition and Gaussian mixture model

Abstract

Manipulating spectral structure often leads to degradation of speech quality, which is mainly due to insufficient smoothness of the modified spectra between frames, and ineffective spectral modification. This paper presents a new spectral modification method to improve the quality of modified speech. If frames are processed independently, discontinuous features may be generated. Therefore, a speech analysis technique called temporal decomposition (TD), which decomposes speech into event targets and event functions, is used to model the spectral evolution effectively. Instead of modifying the speech spectra frame by frame, we only need to modify event targets and event functions. This feature leads to easy modification of the speech spectra, and the smoothness of modified speech is ensured by the shape of event functions. To improve spectral modification, we explore Gaussian mixture model parameters (spectral-GMM parameters) to model the spectral envelope of each event target, and develop a new algorithm for modifying spectral-GMM parameters in accordance with formant scaling factors. We first evaluate the effectiveness of our proposed method in spectra modeling, and then apply it to two areas which require different amounts of spectral modification, emotional speech synthesis and voice gender conversion. Experimental results show that the effectiveness of our proposed method is verified for spectra modeling and spectral modification.

Towards Parametric Speech Synthesis Using Gaussian-Markov Model of Spectral Envelope and Wavelet-Based Decomposition of F0

“…This can yield better fits and smoother formant trajectories. After that, we approximate the modified spectral envelope with a GMM [22]. The GMM parameters are estimated by minimizing a loss function of the observed spectral envelope , and the GMM approximated one expressed by…”

Acoustical Analyses of Tendencies of Intelligibility in Lombard Speech with Different Background Noise Levels

“…1). Formants: The frequencies, bandwidths and amplitudes at F1, F2 and F3 were estimated by linear predictive coding (LPC) and spectral Gaussian mixture model based spectra (spectral-GMM) [8]. F1 and F2 were used to produce the vowel space.…”

Non-negative temporal decomposition of speech parameters

“…Then, the event functions and the event timings are determined by DP and the NMF update rules of Eqs. (7)- (8). Note that the event vectors are not updated at this stage.…”

“…Temporal decomposition (TD) [2] can represent speech parameters as a set of temporally overlapped event functions and corresponding event vectors. TD has been used for many applications: speech coding [2,3], segmentation of speech signals [4,5], analysis of articulatory parameters [6,7] and modification for the speech spectrum [8] as well as for the speaking rhythm [9]. For these purposes, the event functions should be restricted to the range [0, 1].…”