Auditory scene analysis based on time‐frequency integration of shared FM and AM (II): Optimum time‐domain integration and stream sound reconstruction

Abstract

SUMMARYIn the preceding paper, we have proposed a method for auditory scene analysis, in which the instantaneous frequency, frequency change rate, and amplitude change rate in time-frequency space are intensified into a multipeak probability density distribution by voting method and the grouping into streams of mixed sounds is realized. In this paper, as the main point of the second half of this method, we will introduce the assumption that the stream parameters vary slowly according to the known dynamics and propose an integration method on the time axis, in which the probability density distribution of the stream parameters is optimally estimated in time series by a nonparametric Kalman filter. By doing so, the mechanism of higher auditory scene analysis such as enhancement of the accuracy of the stream parameters, interpolation and connection of the breaks of the streams, and introduction of a priori knowledge into stream selection can be realized. Moreover, the separation and reconstruction system of sounds which correspond to streams is constructed, and the proposed technique is verified by fundamental experiments for synthesized sounds or musical sounds and voices.

Auditory scene analysis based on time‐frequency integration of shared FM and AM (I): Lagrange differential features and frequency‐axis integration

“…For this reason, in the proposed method, the filter is a narrow band, and concretely, 1/8 oct is adopted. Short time exception of this condition such as crossing or neighboring of the harmonic components of different sounds is made harmless in the stage of time-axis integration to be reported in the succeeding paper [21].…”

“…In the image processing, the Hough transform [25] used for the detection of lines also has the same structure essentially. In this paper, we will deal only with the integration of the frequency axis which will not be affected by the time variation of the stream, and the time-axis integration will be dealt with in the succeeding paper [21].…”

Flexible Harmonic Temporal Structure for Modeling Musical Instrument

“…Goto presented a method to track F0 of objective single sound from polyphonic musical signals without restriction of the number of simultaneous sounds [2]. Some other multipitch analyzers such as graphical modelbased [3], filterbank-based [4], nonparametric Kalman filtering-based [5], [6]. Then a method for multipitch analysis called Harmonic-Temporal Clustering (HTC) was proposed [7] to deal with the harmonic and temporal structures in both time and frequency directions and shows high performance.…”

Multi-pitch detection algorithm using constrained Gaussian mixture model and information criterion for simultaneous speech

“…Until now, numerous multi-pitch detection methods have been reported not only in speech signal processing [1,2] but also in musical signal processing [3,4,5] and auditory scene analysis [6,7]. Chazan et al addressed a speech separation method by introducing a time warped signal model which allows a continuous pitch variations within a long analysis frame [1].…”