One Shot Learning for Speech Separation

Abstract

Despite the recent success of speech separation models, they fail to separate sources properly while facing different sets of people or noisy environments. To tackle this problem, we proposed to apply meta-learning to the speech separation task. We aimed to find a meta-initialization model, which can quickly adapt to new speakers by seeing only one mixture generated by those people. In this paper, we use model-agnostic meta-learning(MAML) algorithm and almost no inner loop(ANIL) algorithm in Conv-TasNet to achieve this goal. The experiment results show that our model can adapt not only to a new set of speakers but also noisy environments. Furthermore, we found out that the encoder and decoder serve as the feature-reuse layers, while the separator is the task-specific module.

Multi-accent Speech Separation with One Shot Learning

“…described in (Wu et al, 2020). We select at most 12 speakers for each accent and generate speech mixtures for each pair of speakers with the same accents.…”

“…Nonetheless, there is not much work that applied meta-learning on the speech separation task. In our previous work, (Wu et al, 2020), we first proposed to solve the speech separation problem with meta-learning. Their setting is viewing utterance mixtures of two different speakers as a meta task.…”