Classification of general audio data for content-based retrieval

Abstract

In this paper, we address the problem of classi®cation of continuous general audio data (GAD) for content-based retrieval, and describe a scheme that is able to classify audio segments into seven categories consisting of silence, single speaker speech, music, environmental noise, multiple speakers' speech, simultaneous speech and music, and speech and noise. We studied a total of 143 classi®cation features for their discrimination capability. Our study shows that cepstralbased features such as the Mel-frequency cepstral coe cients (MFCC) and linear prediction coe cients (LPC) provide better classi®cation accuracy compared to temporal and spectral features. To minimize the classi®cation errors near the boundaries of audio segments of di erent type in general audio data, a segmentation±pooling scheme is also proposed in this work. This scheme yields classi®cation results that are consistent with human perception. Our classi®cation system provides over 90% accuracy at a processing speed dozens of times faster than the playing rate. Ó

A Bayesian framework for video affective representation

“…This audio histogram was computed from a randomly selected 30 minutes segment of each movie's audio stream. [20] Silence ratio Proportion of silence in a time window [24] After removing silence, the remaining audio signals were classified by a SVM with a polynomial kernel, using the LIBSVM toolbox. (http://www.csie.ntu.edu.tw/~cjlin/libsvm/).…”

Personalization in Multimodal Music Retrieval

“…A good overview of common extraction techniques is presented in [7]. Music content-based features may be low-level representations that stem directly from the audio signal, for example zero-crossing rate [18], amplitude envelope [5], bandwidth and band energy ratio [37], or spectral centroid [67]. Alternatively, audio-based features may be derived or aggregated from low-level properties, and therefore represent aspects on a higher level of music understanding.…”

Multisensor Fusion in Smartphones for Lifestyle Monitoring

“…Let us assume that this is represented by s(t) and the time duration for which the signal is processed is [t 1 , t 2 ]. We extract spectral shape features that are related to the audio content [7]. The cepstral coefficients (c) are defined as follows:…”

“…These "cepstral coefficients" help in differentiating between different classes of dynamic activities (e.g., "Cooking" and "Hygiene"), or different classes of static activities (e.g., "Meeting" and "Driving"). We use the first 12 cepstral coefficients as they are well known to be the most useful in describing the content of an audio signal [7]. Figure III shows an example of how the spectral shape, computed using cepstral coefficients, is significantly different between "Cooking" and "Driving".…”