Development of a statistical parametric synthesis system for operatic singing in German

Abstract

In this paper we describe the development of a Hidden Markov Model (HMM) based synthesis system for operatic singing in German, which is an extension of the HMM-based synthesis system for popular songs in Japanese and English called "Sinsy". The implementation of this system consists of German text analysis, lexicon and Letter-To-Sound (LTS) conversion, and syllable duplication, which enables us to convert a German MusicXML input into context-dependent labels for acoustic modelling. Using the front-end, we develop two operatic singing voices, female mezzo-soprano and male bass voices, based on our new database, which consists of singing data of professional opera singers based in Vienna. We describe the details of the database and the recording procedure that is used to acquire singing data of four opera singers in German. For HMM training, we adopt a singer (speaker)-dependent training procedure. For duration modelling we propose a simple method that hierarchically constrains note durations by the overall utterance duration and then constrains phone durations by the synthesised note duration. We evaluate the performance of the voices with two vibrato modelling methods that have been proposed in the literature and show that HMM-based vibrato modelling can improve the overall quality.

Deep Learning for Singing Processing: Achievements, Challenges and Impact on Singers and Listeners

“…This in contrast to deep learning approaches, which can jointly model timbre and F0 from a set of natural songs. Other machine learning approaches, such as those based on Hidden Markov Models (HMMs) (Saino et al, 2006;Oura et al, 2010;Nakamura et al, 2014;Pucher et al, 2016;Li & Wang, 2016), offer similar flexibility in this aspect, but tend to have other drawbacks. Notably, HMM-based methods tend to have a series of limitations inherent to the model itself, such as modeling each phoneme as a small number of discrete states with constant statistics, and many others.…”