A framework for audio analysis based on classification and temporal segmentation

Abstract

Existing audio tools handle the increasing amount of computer audio data inadequately. The typical taperecorder paradigm for audio interfaces is inflexible and time consuming, especially for large data sets. On the other hand, completely automatic audio analysis and annotation is impossible using current techniques.Alternative solutions are semi-automatic user interfaces that let users interact with sound in flexible ways based on content. This approach offers significant advantages over manual browsing, annotation and retrieval. Furthermore, it can be implemented using existing techniques for audio content analysis in restricted domains. This paper describes a framework for experimenting, evaluating and integrating such techniques. As a test for the architecture, some recently proposed techniques have been implemented and tested. In addition, a new method for temporal segmentation based on audio texture is described. This method is combined with audio analysis techniques and used for hierarchical browsing, classification and annotation of audio files.

Environmental Sound Recognition Using Spectrogram Image Features

“…Lamel et al [8] describes endpoint detector for isolated word segmentation, where a histogram of the low 10db of log energy levels are considered to estimate the background noise. Tzanetakis and Cook [9] describe a methodology for temporal segmentation using different features such as Spectral features, MFCCs, LPC coefficients and pitch. Jasmine et al [10] model silence and noise parameters by assuming that the first 200ms of the speech signal contain noise.…”

Feature extraction for speech and music discrimination

“…The linear prediction coefficients can use the Levinson-Durbin recursion to solve the normal equations that arise from the least-squares formulation [8].…”

Audio thumbnailing of popular music using chroma-based representations

“…This yields a model for frequency that is structurally equivalent to Shepard's decomposition of pitch, such that frequency is also decomposed as (2) where we again restrict and . Alternately, we can calculate chroma from a given frequency using (3) where denotes the greatest integer function. Thus, chroma is simply the fractional part of the base-2 logarithm of frequency.…”

“…We make use of a pattern recognition framework for audio streams, in which the signal is segmented into frames and each frame is described by a set of features. The complexity of the features used varies by application; some commonly used features are described in [3]. This feature-based approach has been applied to general sound classification [4], speech/music discrimination [5], and musical instrument identification [6].…”