Joint time-frequency scattering for audio classification

Abstract

We introduce the joint time-frequency scattering transform, a time shift invariant descriptor of time-frequency structure for audio classification. It is obtained by applying a twodimensional wavelet transform in time and log-frequency to a time-frequency wavelet scalogram. We show that this descriptor successfully characterizes complex time-frequency phenomena such as time-varying filters and frequency modulated excitations. State-of-the-art results are achieved for signal reconstruction and phone segment classification on the TIMIT dataset.

The State of the Art Ten Years After a State of the Art: Future Research in Music Information Retrieval

“…We use the SPGL1 solver [127] with at most 200 iterations, and 2 := 0.01. The second system is MAPsCAT, which uses features computed with the scattering transform [3]. This produces 40 feature vectors of 469 dimensions for a 30-s excerpt.…”

“…We use the Echo Nest Musical Fingerprinter (ENMFP) 3 to generate a fingerprint of every excerpt in GTZAN and to query the Echo Nest database having over 30,000,000 songs. The second column of Table 1 shows that this identifies only 60.6% of the excerpts.…”

“…For Classify, most works measure MGR performance by classification accuracy (the ratio of "correct" predictions to all observations) computed from k-fold stratified cross-validation (kfCV), e.g., 2fCV (4 papers) [7,22,23,56], 3fCV (3 papers) [18,71,74], 5fCV (6 papers) [3,13,30,31,53,100], and 10fCV (55 papers) [2,5,9,11,14,16,17,24-26,28,29,34,35,37,39-42, 44,47-51,57,58,60-64,66-68,70,72,73,75,76,78,79,82-85,88-91,94-96,98,99]. Most of these use a single run of cross-validation; however, some perform multiple runs, e.g., 10 independent runs of 2fCV (10x2CV) [56] or 20x2fCV [22,23], 10x3fCV [71,74], and 10x10fCV [37,70,72,75,[83][84][85].…”

Wavelet transform modulus: phase retrieval and scattering

“…Since its introduction in [8], the scattering transform has found successful applications in, for example, audio genre, visual textures or medical data classification [3,11,12]. …”

Time-scale wavelet scattering using hyperbolic tangent function for vessel sound classification

“…This is similar to MFCC coefficient computation but a scattering-subband filterbank is used The block diagram of the tanh based Scattered Transform Cepstral Coefficients (tanh-STCC) feature extraction algorithm is shown in Figure 2: The amplitude range of recorded sound data is normalized between -1 and 1 [9,13,14] before the filterbank. Pre-emphasis, framing, windowing, logarithm and the DCT block are the same as the ordinary MFCC computation.…”