Reverberation Modeling for Source-Filter-Based Neural Vocoder

Abstract

This paper presents a reverberation module for source-filterbased neural vocoders that improves the performance of reverberant effect modeling. This module uses the output waveform of neural vocoders as an input and produces a reverberant waveform by convolving the input with a room impulse response (RIR). We propose two approaches to parameterizing and estimating the RIR. The first approach assumes a global time-invariant (GTI) RIR and directly learns the values of the RIR on a training dataset. The second approach assumes an utterance-level time-variant (UTV) RIR, which is invariant within one utterance but varies across utterances, and uses another neural network to predict the RIR values. We add the proposed reverberation module to the phase spectrum predictor (PSP) of a HiNet vocoder and jointly train the model. Experimental results demonstrate that the proposed module was helpful for modeling the reverberation effect and improving the perceived quality of generated reverberant speech. The UTV-RIR was shown to be more robust than the GTI-RIR to unknown reverberation conditions and achieved a perceptually better reverberation effect.

APNet: An All-Frame-Level Neural Vocoder Incorporating Direct Prediction of Amplitude and Phase Spectra

“…In our previous work, we proposed the HiNet vocoder [37] and its variant [39]. We have also successfully applied the HiNet vocoder in the reverberation modeling task [40] and denoising and dereveberation task [41], [42], respectively. As shown in Figure 1, the HiNet vocoder uses an ASP and a PSP to predict the frame-level log amplitude spectrum and phase spectrum of a waveform, respectively.…”