Parametric model of spectral envelope to synthesize realistic intensity variations in singing voice

Abstract

In this paper, we propose a method to synthesize the natural variations of spectral envelope as intensity varies in singing voice. To this end, we propose a parametric model of spectral envelope based on novel 4-pole resonators as formant filters. This model has been used to analyse 60 vowels sung at different intensities in order to define a set of functions describing the global variations of parameters along intensity. These functions have been used to modify the intensity of 16 recorded vowels and 8 synthetic vowels generated with Vocaloid. The realism of the transformations performed with our approach has been evaluated by four amateur musicians in comparison to Melodyne for real sounds and to Vocaloid for synthetic sounds. The proposed approach has been proved to achieve more realistic sounds than Melodyne and Vocaloid, especially for loud-to-weak transformations.

Spectral Envelope Transformation in Singing Voice for Advanced Pitch Shifting

“…We did not find evidence of any noticeable modification in the case of other parameters, like gain or slope depth. Moreover, we found no meaningful relationship between the bandwidth of the different resonators in our model and the pitch, although previous works could make us hypothesize the opposite [14]; therefore, the variation of the bandwidth of the resonators has not been considered in our model.…”

“…An indispensable requirement for using a reliable model is to ease the control of parameters; therefore, we opted for a formant-based model developed in a previous work [14]. The spectral envelope of this model is inspired by some speech/singing synthesis systems, like [18,19], and it is based on several resonator filters in parallel determining the acoustic formants, plus a certain slope serving as the glottal source.…”

“…The envelope estimation process can be done manually by visually modifying the parameters of the model to fit the envelope defined by the model to the spectral data [14]; however, this is a tedious and slow process that in the end relies on the judgment of the operator. Therefore, in order to perform the spectral estimation task, we have developed an algorithm to automatically find the parameters of the model by means of an iterative non-linear constrained optimization process.…”

“…Depending on the pitch of the note, trained singers position their voices differently when singing, producing various vocal resonances in the same vocal range, like the chest voice or the falsetto The manuscript is organized as follows: Section 2 describes the first stage, in which a parametric model [14] is used to fit accurately the spectral envelope of a number of recorded vowel sounds. In Section 3, the usage of this tool is explained and used for the analysis of a dataset consisting in 76 sung vowels, in order to obtain a set of functions describing the variations of parameters along pitch.…”

A Mouth Opening Effect Based on Pole Modification for Expressive Singing Voice Transformation

“…[5] proposes a mean to add missing high-frequencies harmonics in the spectrum for weak-to-loud transformations, and compares morphing and spectral modelling approaches for modifying the spectral tilt and F1, thus including an effect of the VTF. In [19], authors also employ a spectral modelling approach to apply intensity transformations of singing voice, using a parametric model of spectral envelope based on 4-pole resonators to modify gain, spectral tilt, and formants' frequencies and bandwidths, based on regressions computed from 60 recorded vowels. However, it requires to properly extract the parameters of all formants, which is not straightforward without manual correction.…”