Large-scale content-based audio retrieval from text queries

Abstract

In content-based audio retrieval, the goal is to find sound recordings (audio documents) based on their acoustic features. This content-based approach differs from retrieval approaches that index media files using metadata such as file names and user tags.In this paper, we propose a machine learning approach for retrieving sounds that is novel in that it (1) uses free-form text queries rather sound sample based queries, (2) searches by audio content rather than via textual meta data, and (3) can scale to very large number of audio documents and very rich query vocabulary.We address the problem of handling generic sounds including a wide variety of sound effects, animal vocalizations and natural scenes. We test a scalable approach based on a passive-aggressive model for image retrieval (PAMIR), and compare it to two state-of-the-art approaches; Gaussian mixture models (GMM) and support vector machines (SVM).We test our approach on two large real-world datasets: a collection of short sound effects, and a noisier and larger collection of user-contributed user-labeled recordings (25K files, 2000 terms vocabulary). We find that all three methods achieved very good retrieval performance. For instance, a positive document is retrieved in the first position of the ranking more than half the time, and on average there are more than 4 positive documents in the first 10 retrieved, for both datasets. PAMIR completed both training and retrieval of all data in less than 6 hours for both datasets, on a single machine. It was one to three orders of magnitude faster than the competing approaches. This approach should therefore scale to much larger datasets in the future.

Robust Sound Event Classification Using Deep Neural Networks

“…These range from parametric signal processing-based approaches [7]- [9] to automatic speech recognition (ASR) inspired methods [10] which often make use of mel-frequency cepstral coefficients (MFCCs) [11] and similar features. One promising new approach uses time-frequency domain spectrogram image features (SIF), introduced by Jonathan Dennis et.…”

Indexing and Retrieval of Music using Gaussian Mixture Model Techniques

“…An average spectrum vector is calculated and the derivation of an estimate for tuning derivation is done by stimulating the filter bank shifts using weighted binning techniques [1]. The pitch representation is performed by the decomposition of a given music signal on 88 frequency bands with centre frequencies corresponding to the pitches A0 to C8 (MIDI pitches p=21 to p=108) in order to extract the chroma features.…”

“…Retrieval of multimedia system information is totally different from retrieval of structured information. Music audio retrieval is normally done by annotating the media with text and uses text based retrieval systems to perform music retrieval [1]. But when the music information is voluminous, text based annotation becomes a tedious job.…”

Evaluating multimedia features and fusion for example-based event detection

“…The BOAW has recently been used for audio document retrieval [35] and copy detection [36], as well as MED tasks [37]. Our recent work [38] describes the basic BOAW approach.…”