Shifted and convolutive source-filter non-negative matrix factorization for monaural audio source separation

Nakamura, Tomohiko; Kameoka, Hirokazu

doi:10.1109/icassp.2016.7471723

Cited by 5 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our knowledge, this is the first time in the literature to reveal the flaws and limitations of the excitation-filter product representation assumed in the CWT domain. Although the NMF-based model presented in [18] also uses the excitation-filter product representation in the CWT domain, we did not consider this model for comparison since it is for supervised settings.…”

Section: B Source-filter Model Representation In Cwt Domainmentioning

confidence: 99%

See 1 more Smart Citation

Harmonic-Temporal Factor Decomposition for Unsupervised Monaural Separation of Harmonic Sounds

Nakamura

Kameoka

2021

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

We address the problem of separating a monaural mixture of harmonic sounds into the audio signals of individual semitones in an unsupervised manner. Unsupervised monaural audio source separation has thus far been mainly addressed by two approaches: one rooted in computational auditory scene analysis (CASA) and the other based on non-negative matrix factorization (NMF). These approaches focus on different clues for making source separation possible. A CASA-based method called harmonic-temporal clustering (HTC) focuses on a local time-frequency structure of individual sources, whereas NMF focuses on a global time-frequency structure of music spectrograms. These clues do not conflict with each other and can be used to achieve a more reliable audio source separation algorithm. Hence, we propose a monaural audio source separation framework, harmonic-temporal factor decomposition (HTFD), by developing a spectrogram model that encompasses the features of the models used in the NMF and HTC approaches. We further incorporate a source-filter model to build an extension of HTFD, source-filter HTFD (SF-HTFD). We derive efficient parameter estimation algorithms of HTFD and SF-HTFD based on the auxiliary function principle. We show, through music source separation experiments, the efficacy of HTFD and SF-HTFD compared with conventional methods. Furthermore, we demonstrate the effectiveness of HTFD and SF-HTFD for automatic musical key transposition.

show abstract

Section: B Source-filter Model Representation In Cwt Domainmentioning

confidence: 99%

“…Several attempts have already been made to incorporate the source-filter model into NMF to enhance the performance of audio source separation and multipitch analysis [11]- [18]. In these studies, a spectrum of each source is simply represented as the product of excitation and filter spectra.…”

Section: Introductionmentioning

confidence: 99%

Harmonic-Temporal Factor Decomposition for Unsupervised Monaural Separation of Harmonic Sounds

Nakamura

Kameoka

2021

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Though it is more complicated, 2DNMF is a more compact representation than 1D convolutional NMF (in time only), in which it is necessary to store a different template for each pitch shift of each instrument. We note that pitch shifts in real instruments are more complicated than shifting all frequency bins by the same perceptual amount [14], but the basic version of 2DNMF is fine for our purposes.…”

Section: Nmf2d For Joint Blind Factorization / Filteringmentioning

confidence: 99%

Cover Song Synthesis by Analogy

Tralie¹

2018

Preprint

View full text Add to dashboard Cite

In this work, we pose and address the following "cover song analogies" problem: given a song A by artist 1 and a cover song A' of this song by artist 2, and given a different song B by artist 1, synthesize a song B' which is a cover of B in the style of artist 2. Normally, such a polyphonic style transfer problem would be quite challenging, but we show how the cover songs example constrains the problem, making it easier to solve. First, we extract the longest common beat-synchronous subsequence between A and A', and we time stretch the corresponding beat intervals in A' so that they align with A. We then derive a version of joint 2D convolutional NMF, which we apply to the constant-Q spectrograms of the synchronized segments to learn a translation dictionary of sound templates from A to A'. Finally, we apply the learned templates as filters to the song B, and we mash up the translated filtered components into the synthesized song B' using audio mosaicing. We showcase our algorithm on several examples, including a synthesized cover version of Michael Jackson's "Bad" by Alien Ant Farm, learned from the latter's "Smooth Criminal" cover.

show abstract

“…The work of 2-D deconvolution NMF, Kirbiz and Gunsel [72] proposed to improve the perceptual qualities of separated STFT sources by means of a defined perceptual evaluation of audio quality (PEAQ) auditory model [73]. Based on the source-filter theory, Tomohiko and Hirokazu [74] describe the spectrogram of a mixture signal as the sum of the products between the shifted copies of excitation spectrum templates and filter spectrum templates to reduce the separation error caused by using a shifted copy of a spectrum template to represent the spectra of different fundamental frequency F 0 s. They developed the shifted NMF model in the form of…”

Section: Shift Constraints On Wmentioning

confidence: 99%

“…The multiplicative update is not the only approach in solving the iterative optimization problem of NMF; the auxiliary-function method [85] is an alternative method adopted in work [64,74,86,87]. Meanwhile, the distance function D(•) is extended into Kullback-Leibler divergence [88][89][90], Itakura-Saito divergence [91], the developing β-divergence [92][93][94], α-β-divergence [95] and α-β-γ-divergence [96].…”

Section: Auxiliary Function Iterative Algorithmmentioning

confidence: 99%

Monaural Music Separation via Supervised Non-Negative Matrix Factor with Side-Information

Peng¹

View full text Add to dashboard Cite

In this dissertation, a supervised source template nonnegative matrix factorization (NMF) algorithm is proposed to solve the monaural music source separation problem. Different from the previous state-of-the-art algorithms, the basic theoretical concept of the proposed algorithm considers the spectrogram from an audio mixture as linear combinations of note templates. Having prior knowledge of these note templates for each source, we can estimate and determine the activities of each template in recordings to build a mask of each source. Through the masks, the audio of target tracks can be reconstructed. We reviewed previous research on source separation for monaural music audio separation and compared these work with our proposed algorithm not only in mathematical expressions but also in separation performances. First, the prior knowledge of note templates is informed by musical instrument audio dataset. The spectrograms from these instruments are obtained and factored into a source resonance character matrix and a source impulse excitation matrix by assuming that the spectrum of the different notes are formed by the resonance effects from an impulse excitation. Secondary, according to the prior informed note templates, their onset-offset-like features are estimated by using the multiplicative update rule and supervised by the proposed pitch-checking algorithm to remove misleading estimations. Finally, the supervised note onset-offset-like features alternatively become a constraint to help the proposed model evolve its prior informed note templates into the forms given by the recorded instruments. We employed the TRIOS and the Bach-10 dataset for our multi-source separation performance tests. Among the source separation algorithms, our proposed supervised source template NMF and the state-of-the-art algorithms including the sound-prism and the Oracle-toolbox methods were selected to make comparisons. Furthermore, we added white Gaussian noise into the audio mixture to simulate the background full of the random noise to test the noise characteristics of each algorithm. The iii First of all, I wish to thank Prof. Richard Dansereau, my supervisor, for bringing me into the fantastic world of Artificial Intelligence and Machine Learning, continually giving me generous funding support, confidence and knowledge, and guiding me on this road. Special thanks to Tony Wacheski and Sean Kormilo who are the CEO, Co-Founder and Senior Software Engineer at Anystone Inc., who have opened a gate for me by programming and coding our research findings into commercial software. Finally, my sincere appreciation to my wife, especially for her great love and care which keeps me smiling even in the darkest time of illness. vi

show abstract

Shifted and convolutive source-filter non-negative matrix factorization for monaural audio source separation

Cited by 5 publications

References 21 publications

Harmonic-Temporal Factor Decomposition for Unsupervised Monaural Separation of Harmonic Sounds

Harmonic-Temporal Factor Decomposition for Unsupervised Monaural Separation of Harmonic Sounds

Cover Song Synthesis by Analogy

Monaural Music Separation via Supervised Non-Negative Matrix Factor with Side-Information

Contact Info

Product

Resources

About