LPCNET: Improving Neural Speech Synthesis through Linear Prediction

Abstract

Neural speech synthesis models have recently demonstrated the ability to synthesize high quality speech for text-to-speech and compression applications. These new models often require powerful GPUs to achieve real-time operation, so being able to reduce their complexity would open the way for many new applications. We propose LPCNet, a WaveRNN variant that combines linear prediction with recurrent neural networks to significantly improve the efficiency of speech synthesis. We demonstrate that LPCNet can achieve significantly higher quality than WaveRNN for the same network size and that high quality LPCNet speech synthesis is achievable with a complexity under 3 GFLOPS. This makes it easier to deploy neural synthesis applications on lower-power devices, such as embedded systems and mobile phones.

Improving LPCNET-Based Text-to-Speech with Linear Prediction-Structured Mixture Density Network

“…In detail, the distribution is sharpened by directly reducing the generated scale parameters shown in Eq. (5). Because the buzziness and the hiss of synthetic speech are sensitive to the sharpness of the distribution, the scale parameters must be adjusted carefully.…”

“…In the LPCNet vocoder, the specification was almost the same as its original version [5]. In the frame-level network, the dimension of the two FC layers was set to 64.…”

“…The term prediction is defined as an intermediate LP operation[5], which will be discussed in detail in section 3.…”

CLCNET: Deep Learning-Based Noise Reduction for Hearing aids using Complex Linear Coding

“…Linear predictive coding is known to be a suitable model of the vocal tract response [15] and is still used in SOTA approaches for speech coding and synthesis [14]. Given a signal x k at sample k, LPC can be described as the linear combinationx…”

“…LPC is able to perfectly model a superposition of multiple sinusoidals given enough coefficients. Due to this property, LPC finds a use case in speech coding and synthesis [14]. Yet, it is often only applied on time-domain signals as a post-processing step.…”