Online score‐informed source separation in polyphonic mixtures using instrument spectral patterns

Abstract

Soundprism is a real‐time algorithm to separate polyphonic music audio into source signals, given the musical score of the audio in advance. This paper presents a framework for a Soundprism implementation. A study of the sound quality of the online score‐informed source separation is shown, although a real‐time implementation is not carried out. The system is compound of two stages: (1) a score follower that matches a MIDI score position to each time frame of the musical performance; and (2) a source separator based on a nonnegative matrix factorization approach guided by the score. Real audio mixtures composed of an instrumental quartets were employed to obtain preliminary results of the proposed system.

Online/offline score informed music signal decomposition: application to minus one

“…Although several online audio-to-score approaches have been developed in the literature [22,28,29], only the works in [6,22,30] combined score alignment with SS in an online fashion. In [22] and in its extension [6], the alignment is performed using a hidden Markov process model, where each audio frame is associated with a 2-D state of score position and tempo.…”

“…Regarding the SS, [22] uses a soft masking strategy based on a multipitch estimator, whereas the method in [6] used a frame-level NMF with a trained dictionary which is updated during the factorization. On the other hand, in our preliminary work [30], we proposed to use the source separation procedure presented in [13] along with the real-time implementation of the online alignment method from [31] presented in [32]. The signal model used in [30] was the same than in [13] but restricted to single-channel signals.…”

“…On the other hand, in our preliminary work [30], we proposed to use the source separation procedure presented in [13] along with the real-time implementation of the online alignment method from [31] presented in [32]. The signal model used in [30] was the same than in [13] but restricted to single-channel signals. However, with this original signal model, the separation performance was unreliable when dealing with large ensembles datasets.…”

“…In this paper, we propose a score-informed SS framework, which suits for both online and offline applications, based on NMF and DTW. Similar to [6,30], we use a priori learned dictionary composed of spectral templates (a.k.a basis functions) for the instruments presented in the score. However, in this work, the score information is encoded within the signal model under the assumption that a music signal can be described as a sequence of unique occurrences of individual and concurrent notes from several instruments (here denoted as score units).…”

“…Unlike a previous work presented by the authors in [29,30], where synthetic signals generated from the MIDI score are used to learn spectral patterns for the alignment process, here, we use isolated notes of real solo instrument recordings to learn a dictionary of spectral patterns. Using this dictionary of isolated notes allows to model the relative amplitude contribution of each instrument in order to obtain a velocity model for each note and instrument.…”