Soft context clustering for F0 modeling in HMM-based speech synthesis

Abstract

This paper proposes the use of a new binary decision tree, which we call a soft decision tree, to improve generalization performance compared to the conventional 'hard' decision tree method that is used to cluster context-dependent model parameters in statistical parametric speech synthesis. We apply the method to improve the modeling of fundamental frequency, which is an important factor in synthesizing natural-sounding high-quality speech. Conventionally, hard decision tree-clustered hidden Markov models (HMMs) are used, in which each model parameter is assigned to a single leaf node. However, this 'divide-and-conquer' approach leads to data sparsity, with the consequence that it suffers from poor generalization, meaning that it is unable to accurately predict parameters for models of unseen contexts: the hard decision tree is a weak function approximator. To alleviate this, we propose the soft decision tree, which is a binary decision tree with soft decisions at the internal nodes. In this soft clustering method, internal nodes select both their children with certain membership degrees; therefore, each node can be viewed as a fuzzy set with a context-dependent membership function. The soft decision tree improves model generalization and provides a superior function approximator because it is able to assign each context to several overlapped leaves. In order to use such a soft decision tree to predict the parameters of the HMM output probability distribution, we derive the smoothest (maximum entropy) distribution which captures all partial first-order moments and a global second-order moment of the training samples. Employing such a soft decision tree architecture with maximum entropy distributions, a novel speech synthesis system is trained using maximum likelihood (ML) parameter re-estimation and synthesis is achieved via maximum output probability parameter generation. In addition, a soft decision tree construction algorithm optimizing a log-likelihood measure is developed. Both subjective and objective evaluations were conducted and indicate a considerable improvement over the conventional method.

Initialization of Hidden Markov and Semi‐Markov Models: A Critical Evaluation of Several Strategies

“…Hidden (semi)-Markov models are the state of the art for modeling univariate and multivariate time series data (Barbu & Limnios, 2009;Bulla et al, 2010;Yu, 2010;Maruotti, 2011;Bartolucci et al, 2013;Zucchini et al, 2016). Applications can be found in ecology (Bulla et al, 2012;Martinez-Zarzoso & Maruotti, 2013;Mastrantonio et al, 2015;Maruotti et al, 2016Patterson et al, 2017), medicine (Shirley et al, 2010;Maruotti & Rocci, 2012;Langrock et al, 2013;Lagona et al, 2014;Marino et al, 2018;Punzo et al, 2018;2019), finance (Bulla & Bulla, 2006;Bulla, 2011;Ang & Timmermann, 2012;Langrock et al, 2012;Bernardi et al, 2017;Hambuckers et al, 2018;, psychology (Visser, 2011), social sciences (Langrock, 2011;Punzo & M & P Maruotti, 2016;, sport performance (Ötting et al, 2020), speech recognition (Khorram et al, 2015), music modelling (Pikrakis et al, 2006), network performance (Nguyen & Roughan, 2012) and in many more empirical settings (see Yu, 2015, and references therein).…”