Segmentation, Indexing, and Retrieval for Environmental and Natural Sounds

Wichern, Gordon; Xue, Jiachen; Thornburg, Harvey; Mechtley, Brandon; Spanias, Andreas

doi:10.1109/tasl.2010.2041384

Cited by 56 publications

(19 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The images are snapshots of the 3D objects with no background information, however, the framework can be extended to include real images (this implies also that the above 2D shape-based descriptors should be replaced by the appropriate descriptors for real images). Finally, the audio descriptors are extracted using the algorithm presented in [42].…”

Section: Resultsmentioning

confidence: 99%

Multimedia search and retrieval using multimodal annotation propagation and indexing techniques

Lazaridis

Αξενόπουλος

Rafailidis

et al. 2013

Signal Processing: Image Communication

View full text Add to dashboard Cite

In this paper, a novel framework for multimodal search and retrieval of rich media objects is presented. The searchable items are media representations consisting of multiple modalities, such as 2D images, 3D objects and audio files, which share a common semantic concept. A manifold learning technique based on Laplacian Eigenmaps was appropriately modified in order to merge the low-level descriptors of each separate modality and create a new low-dimensional multimodal feature space, where all media objects can be mapped irrespective of their constituting modalities. To accelerate search and retrieval and make the framework suitable even for web-scale applications, a multimedia indexing scheme is adopted, which represents each object of the dataset by the ordering of a number of reference objects. Moreover, the hubness property is introduced in this paper as a criterion to select the most representative reference objects, thus, achieving the maximum possible performance of indexing. The content-based similarity of the multimodal descriptors is also used to automatically annotate the objects of the dataset using a predefined set of attributes. Annotation propagation is utilized to approximate the multimodal descriptors for multimodal queries that do not belong to the dataset.

show abstract

Section: Resultsmentioning

confidence: 99%

Multimedia search and retrieval using multimodal annotation propagation and indexing techniques

Lazaridis

Αξενόπουλος

Rafailidis

et al. 2013

Signal Processing: Image Communication

View full text Add to dashboard Cite

show abstract

“…In [5], [6] authors make use of SED for designing audio context recognition systems, while in [7] and [8] SED is exploited for automatic tagging and audio segmentation respectively. Moreover, SED also found many direct applications in a variety of scenarios, some examples being context-based indexing and retrieval in multimedia databases [9], unobtrusive health monitoring [10], and audio-based surveillance [11]- [13].…”

Section: Introductionmentioning

confidence: 99%

A neural network approach for sound event detection in real life audio

Valenti

Tonelli

Vesperini

et al. 2017

2017 25th European Signal Processing Conference (EUSIPCO)

View full text Add to dashboard Cite

Abstract-This paper presents and compares two algorithms based on artificial neural networks (ANNs) for sound event detection in real life audio. Both systems have been developed and evaluated with the material provided for the third task of the Detection and Classification of Acoustic Scenes and Events (DCASE) 2016 challenge. For the first algorithm, we make use of an ANN trained on different features extracted from the downmixed mono channel audio. Secondly, we analyse a binaural algorithm where the same feature extraction is performed on four different channels: the two binaural channels, the averaged monaural signal and the difference between the binaural channels. The proposed feature set comprehends, along with mel-frequency cepstral coefficients and log-mel energies, also activity information extracted with two different voice activity detection (VAD) algorithms. Moreover, we will present results obtained with two different neural architectures, namely multilayer perceptrons (MLPs) and recurrent neural networks. The highest scores obtained on the DCASE 2016 evaluation dataset are achieved by a MLP trained on binaural features and adaptive energy VAD; they consist of an averaged error rate of 0.79 and an averaged F1 score of 48.1%, thus marking an improvement over the best score registered in the DCASE 2016 challenge.

show abstract

“…Automatic sound event detection can be utilized in a variety of applications, including context-based indexing and retrieval in multimedia databases [3,4], unobtrusive monitoring in health care [5], surveillance [6], and military applications [7]. The symbolic information about the sound events can be used in other research areas, e.g., audio context recognition [8,9], automatic tagging [10], and audio segmentation [11].…”

Section: Introductionmentioning

confidence: 99%

Context-dependent sound event detection

Heittola

Mesaros

Eronen

et al. 2013

J AUDIO SPEECH MUSIC PROC.

200

137

View full text Add to dashboard Cite

The work presented in this article studies how the context information can be used in the automatic sound event detection process, and how the detection system can benefit from such information. Humans are using context information to make more accurate predictions about the sound events and ruling out unlikely events given the context. We propose a similar utilization of context information in the automatic sound event detection process. The proposed approach is composed of two stages: automatic context recognition stage and sound event detection stage. Contexts are modeled using Gaussian mixture models and sound events are modeled using three-state left-to-right hidden Markov models. In the first stage, audio context of the tested signal is recognized. Based on the recognized context, a context-specific set of sound event classes is selected for the sound event detection stage. The event detection stage also uses context-dependent acoustic models and count-based event priors. Two alternative event detection approaches are studied. In the first one, a monophonic event sequence is outputted by detecting the most prominent sound event at each time instance using Viterbi decoding. The second approach introduces a new method for producing polyphonic event sequence by detecting multiple overlapping sound events using multiple restricted Viterbi passes. A new metric is introduced to evaluate the sound event detection performance with various level of polyphony. This combines the detection accuracy and coarse time-resolution error into one metric, making the comparison of the performance of detection algorithms simpler. The two-step approach was found to improve the results substantially compared to the context-independent baseline system. In the block-level, the detection accuracy can be almost doubled by using the proposed context-dependent event detection.

show abstract

Segmentation, Indexing, and Retrieval for Environmental and Natural Sounds

Cited by 56 publications

References 31 publications

Multimedia search and retrieval using multimodal annotation propagation and indexing techniques

Multimedia search and retrieval using multimodal annotation propagation and indexing techniques

A neural network approach for sound event detection in real life audio

Context-dependent sound event detection

Contact Info

Product

Resources

About