PeriodNet: A Non-Autoregressive Raw Waveform Generative Model With a Structure Separating Periodic and Aperiodic Components

Abstract

This paper presents PeriodNet, a non-autoregressive (non-AR) waveform generative model with a new model structure for modeling periodic and aperiodic components in speech waveforms. Non-AR raw waveform generative models have enabled the fast generation of high-quality waveforms. However, the variations of waveforms that these models can reconstruct are limited by training data. In addition, typical non-AR models reconstruct a speech waveform from a single Gaussian input despite the mixture of periodic and aperiodic signals in speech. These may significantly affect the waveform generation process in some applications such as singing voice synthesis systems, which require reproducing accurate pitch and natural sounds with less periodicity, including husky and breath sounds. PeriodNet uses a parallel or series model structure to model a speech waveform to tackle these problems. Two sub-generators connected in parallel or in series take an explicit periodic and aperiodic signal (sine wave and Gaussian noise) as an input. Since PeriodNet models periodic and aperiodic components by focusing on whether these input signals are autocorrelated or not, it does not require external periodic/aperiodic decomposition during training. Experimental results show that our proposed structure improves the naturalness of generated waveforms. We also show that speech waveforms with a pitch outside of the training data range can be generated with more naturalness.

Text to Song Synchronization Using Deep Neural Network

“…As we all know, the text is converted into lyrics and sung by the computer from musical scores using the SVS system based on HMM, but because the voice is easily distinguishable from the natural singing voice, we can also implement the SVS using DNN, and DNN-based systems provide more utilized and efficient results when compared to HMM-based systems. It also outperforms the HMMbased system in subjective listening [10]. The PeriodNet system, which converts/models periodic and aperiodic waveforms into speech waveforms, can also be implemented.…”

Harmonic-Net: Fundamental Frequency and Speech Rate Controllable Fast Neural Vocoder

“…In the case of neural vocoders, the synthesis quality deteriorates when the input f o is not included in the range of the training data. Several approaches have been proposed to solve this problem [33], [34], [35], [36], [37], [38], [39]. In contrast to AR models [33], [34], [35], non-AR models [36], [37], [38], [39] can realize real-time inference.…”

“…Several approaches have been proposed to solve this problem [33], [34], [35], [36], [37], [38], [39]. In contrast to AR models [33], [34], [35], non-AR models [36], [37], [38], [39] can realize real-time inference. The neural source filter [36] introduces nonlinear filtering and dilated convolutional layers for parametrically generated source excitation signals corresponding to f o by source-filter modeling [40].…”

“…The explicit input of the excitation signal into a generator based on source-filter modeling is highly effective for improving the control accuracy of f o . PeriodNet [39] uses sinusoidal waves and white noise as excitation signals to explicitly separate the generation of periodic waveforms from that of aperiodic waveforms. This approach is particularly effective for singing voice synthesis because the large f o fluctuations contained in the singing voice can be efficiently captured by the input of the excitation signals.…”

“…Harmonic-Net and Harmonic-Net+ are then proposed in Section III. Section IV describes experiments to compare Harmonic-Net and Harmonic-Net+ with several conventional methods: WORLD [8], WaveNet [50], HiFi-GAN [21], uSFGAN [38], and PeriodNet [39]. Finally, conclusions are presented in Section V.…”