FBWave: Efficient and Scalable Neural Vocoders for Streaming Text-To-Speech on the Edge

Abstract

Nowadays more and more applications can benefit from edge-based text-to-speech (TTS). However, most existing TTS models are too computationally expensive and are not flexible enough to be deployed on the diverse variety of edge devices with their equally diverse computational capacities. To address this, we propose FBWave, a family of efficient and scalable neural vocoders that can achieve optimal performance-efficiency trade-offs for different edge devices. FBWave is a hybrid flow-based generative model that combines the advantages of autoregressive and nonautoregressive models. It produces high quality audio and supports streaming during inference while remaining highly computationally efficient. Our experiments show that FB-Wave can achieve similar audio quality to WaveRNN while reducing MACs by 40x. More efficient variants of FBWave can achieve up to 109x fewer MACs while still delivering acceptable audio quality. Audio demos are available at https: //bichenwu09.github.io/vocoder_demos.

WOLONet: Wave Outlooker for Efficient and High Fidelity Speech Synthesis

“…The gigantic size of the model also limits its ability to compute in real-time. Another notable attempt in this area is flow-based models such as WaveGlow [15], FloWavenet [16], Melflow [17], WaveFlow [18], and FBWAVE [19], etc. They use a single loglikelihood loss to train specially designed invertible models.…”

“…They use a single loglikelihood loss to train specially designed invertible models. The inference speed is faster than AR models and can even be used on mobile devices after additional development efforts on CPU [19]. However, the unstable training process and unsatisfactory synthesis quality prevent it from being used in industrial applications.…”

Improve GAN-based Neural Vocoder using Pointwise Relativistic LeastSquare GAN

“…The gigantic size of their model also restricts it from achieving real-time computation. Another notable endeavor in this field is made by flow-based models including waveglow [12], flowavenet [13], Melflow [14], waveflow [15] and FBWAVE [16] etc. They apply a single log-likelihood loss to train specially designed invertible models.…”

“…They apply a single log-likelihood loss to train specially designed invertible models. The inference speed is faster than AR models and can be deployed even on mobile CPU after extra efforts in engineering [16]. However, its unstable training process and unsatisfying synthesis quality prevent it from being deployed in industrial applications.…”