DeepF0: End-To-End Fundamental Frequency Estimation for Music and Speech Signals

Abstract

We propose a novel pitch estimation technique called DeepF0, which leverages the available annotated data to directly learns from the raw audio in a data-driven manner. f0 estimation is important in various speech processing and music information retrieval applications. Existing deep learning models for pitch estimations have relatively limited learning capabilities due to their shallow receptive field. The proposed model addresses this issue by extending the receptive field of a network by introducing the dilated convolutional blocks into the network. The dilation factor increases the network receptive field exponentially without increasing the parameters of the model exponentially. To make the training process more efficient and faster, DeepF0 is augmented with residual blocks with residual connections. Our empirical evaluation demonstrates that the proposed model outperforms the baselines in terms of raw pitch accuracy and raw chroma accuracy even using 77.4% fewer network parameters. We also show that our model can capture reasonably well pitch estimation even under the various levels of accompaniment noise.

HarmoF0: Logarithmic Scale Dilated Convolution For Pitch Estimation

“…The proposed model is compared with 4 state-of-the-art methods: pYin [7], SWIPE [6], CREPE [8], and DeepF0 [3]. The performance in noisy conditions and ablation study are also discussed in this section.…”

“…But the CREPE model, with parameters up to 22.2M, is computationally expensive. To reduce the parameters and speed up the calculation, the DeepF0 [3] is proposed by applying dilated convolution and skip connections, achieving the state-of-the-art on the tested datasets. However, as a compromise, the DeepF0 does not perform as well as the CREPE in loud noisy environments.…”

“…And based on this approach, we construct HarmoF0, a fully convolutional network for pitch estimation. Empirical evaluation demonstrates that our model outperforms the state-ofthe-art DeepF0 [3], but with much fewer parameters. In addition, in noisy environments, it still surpasses all the baselines, including DeepF0, SWIPE [6], pYin [7], and CREPE [8], revealing a high capability of noise resistance.…”

“…Some researchers try to use the raw waveform as the input of the neural networks. The DeepF0 [3] feeds the raw audio sample to the model and applies dilated convolution [4] to estimate pitch. It achieved high performance, but the CNN model working in the time *Yi Yu and Wei Li are the corresponding authors.…”

Improved Meta Learning for Low Resource Speech Recognition

“…Modern deep learning based end-to-end (E2E) models have lately become extremely popular in the speech community [1] and have achieved a significant milestone in terms of performance. These systems have been deployed under commercial domains as they have shown consistently lower word error rates that are close to 1-2% [2].…”