Underdetermined reverberant acoustic source separation using weighted full-rank nonnegative tensor models

Abstract

In this paper, a fusion of K models of full-rank weighted nonnegative tensor factor two-dimensional deconvolution (K-wNTF2D) is proposed to separate the acoustic sources that have been mixed in an underdetermined reverberant environment. The model is adapted in an unsupervised manner under the hybrid framework of the generalized expectation maximization and multiplicative update algorithms. The derivation of the algorithm and the development of proposed full-rank K-wNTF2D will be shown. The algorithm also encodes a set of variable sparsity parameters derived from Gibbs distribution into the K-wNTF2D model. This optimizes each sub-model in K-wNTF2D with the required sparsity to model the time-varying variances of the sources in the spectrogram. In addition, an initialization method is proposed to initialize the parameters in the K-wNTF2D. Experimental results on the underdetermined reverberant mixing environment have shown that the proposed algorithm is effective at separating the mixture with an average signal-to-distortion ratio of 3 dB.

Hybrid Deep Learning Model for Singing Voice Separation

“…Y.-H. Yang [29] employed dictionary learning methods to estimate the subspace structures of musical sources and introduced a new approach called multiple low-rank representations (MLRR) that is used for decomposition by the learnt dictionaries. Al Tmem et al [2,3,22,27] proposed an algorithm based on the factorization method with a hybrid framework that combined both the multiplicative update and the Expectation-Maximization algorithms. Wang et al [26] learned the ideal binary mask using deep neural networks, and source separation issues were regarded as binary classification problems.…”

Audio-Visual Source Separation Based Fusion Techniques

“…Early works for speech separation solely utilized the audio signal [3][4][5][6][7]. The audio only speech separation problem is inherently challenging due to its ambiguity, making it difficult to achieve satisfactory outcomes without additional information, for example, prior knowledge or certain microphone configuration.…”

An Overview of Audio-Visual Source Separation Using Deep Learning

“…The process of distinguishing particular audio sources from a mixture of audio signals using visual indicators as further information is known as audio-visual source separation. This method differs from conventional ones that exclusively rely on the audio stream for source separation [1][2][3][4][5]. In other words, it is a technique that utilizes both auditory and visual information to separate individual sound sources from a mixed audio signal.…”