Phase constrained complex NMF: Separating overlapping partials in mixtures of harmonic musical sources

Abstract

This paper examines complex non-negative matrix factorization (CMF) as a tool for separating overlapping partials in mixtures of harmonic musical sources. Unlike non-negative matrix factorization (NMF), CMF allows for the development of source separation procedures founded on a mixture model rooted in the complexspectrum domain (in which the superposition of overlapping sources is preserved). This paper introduces a physically motivated phase constraint based on the assumption that the source's pitch is sufficient in specifying the phase evolution of the harmonics over time, uniting sinusoidal modelling of acoustic sources with the CMF analysis of their spectral representations. The CMF-based separation procedure, armed with this novel phase constraint, is demonstrated to offer a superior performance to NMF when employed as a tool for separating overlapping partials in the acoustic test cases considered.

Model-Based STFT Phase Recovery for Audio Source Separation

“…The main difference between the PU algorithm and consistency-based approaches is that the former relies on a signal model while the latter exploit a property of the STFT. Such an approach has been used in the phase vocoder algorithm [23] for time stretching, and applied to speech enhancement [24], [25], audio restoration [26] and source separation [27]. In many cases, the mixtures are assumed to be in harmonic proportions, which means that the partial frequencies are integer multiples of a fundamental frequency, but the PU technique can be extended to signals which do not comply with this assumption [26].…”

Complex NMF under phase constraints based on signal modeling: Application to audio source separation

“…For instance, the widely used model of mixtures of sinusoids [9] can lead to explicit constraints for phase reconstruction that exploit the relationships between adjacent TF bins. This approach has been used in the phase vocoder algorithm [10], integrated into a CNMF framework [11] and applied to speech signal reconstruction [12]. In [13], we proposed to generalize this approach and have provided a phase unwrapping algorithm that has been applied to an audio signal restoration task.…”

Reducing Interference with Phase Recovery in DNN-based Monaural Singing Voice Separation

“…Alternatively, one can extract phase constraints from the sinusoidal model, which is widely used for representing audio signals [11,22]. It can be shown [23] that the STFT phase µ of a signal modeled as a sum of sinusoids in the time domain follows the phase unwrapping (PU) equation:…”

“…It has been used in many audio applications, including time stretching [23], speech enhancement [22] and source separation [7,11,24].…”

“…Phase recovery algorithms exploit phase constraints that originate from consistency [8], a property of the STFT that arises from its redundant signal representation, or from a signal model that approximates time-domain signals as a sum of sinusoids [7]. The above mentioned phase constraints have been applied to a source separation task [9,7] and combined with magnitude estimation techniques in order to design full and phase-aware separation systems [10,11]. However, these systems are based on variants of NMF methods, which provides fairly good separation results in scenarios where the sources are well represented with stationary spectral atoms (over time) and uniform temporal activations (over frequencies).…”