A Systematic Evaluation of the Bag-of-Frames Representation for Music Information Retrieval

Su, Li; Yeh, Chin‐Chia Michael; Liu, Jen-Yu; Wang, Ju-Chiang; Yang, Yi-Hsuan

doi:10.1109/tmm.2014.2311016

Cited by 42 publications

(37 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That is to standardize audio data format into Wav format and to split the original audio data into 30 sec long. In work done by Su et al (2014) it is stated that the reasons of using 30 seconds long audio data is to avoid wrong features extraction which may lead to inaccuracy of training and testing result.…”

Section: Methodsmentioning

confidence: 99%

Automatic Music Emotion Classification Using Artificial Neural Network Based on Vocal and Instrumental Sound Timbres

Mokhsin¹,

Rosli²,

Zambri³

et al. 2014

Journal of Computer Science

View full text Add to dashboard Cite

Detecting emotion features in a song remains as a challenge in various area of research especially in Music Emotion Classification (MEC). In order to classify selected song with certain mood or emotion, the algorithms of the machine learning must be intelligent enough to learn the data features as to match the features accordingly to the accurate emotion. Until now, there were only few studies on MEC that exploit audio timbre features from vocal part of the song incorporated with the instrumental part of a song. Timbre features is the quality of a musical features or sound that distinguishes different types of sound production in human voices and musical instruments such as string instruments, wind instruments and percussion instruments. Most of existing works in MEC are done by looking at audio, lyrics, social tags or combination of two or more classes. The question is does exploitation of both timbre features from both vocal and instrumental sound features helped in producing positive result in MEC? Thus, this research present works on detecting emotion features in Malay popular music using artificial neural network by extracting audio timbre features from both vocal and instrumental sound clips. The findings of this research will collectively improve MEC based on the manipulation of vocal and instrumental sound timbre features, as well as contributing towards the literature of music information retrieval, affective computing and psychology.

show abstract

Section: Methodsmentioning

confidence: 99%

Automatic Music Emotion Classification Using Artificial Neural Network Based on Vocal and Instrumental Sound Timbres

Mokhsin¹,

Rosli²,

Zambri³

et al. 2014

Journal of Computer Science

View full text Add to dashboard Cite

show abstract

“…Refs. [18] and [19] show that spectrogram is the best local representation for sparse coding based audio-word type features comparing to Mel-spectrum, MFCC, Sonogram, and Constant-Q transform.…”

Section: Baseline Audio-word Extraction Pipelinementioning

confidence: 99%

“…Late Temporal Pooling: In our previous work [18], [19], [22], we have considered two types of late temporal pooling scheme: Bag-of-Frames (BoF) and Histogram based Bag-ofSegments (HBoS). BoF is the most common and simple way to pool audio-word-type features [14], [16], [22], where the audio-words for a given music clip are summed holistically over the whole clip.…”

Section: Baseline Audio-word Extraction Pipelinementioning

confidence: 99%

“…Among all the features, audio-word has been shown useful in a variety of MIR tasks [13], [14], [15], [16], owing to its ability to represent music information that happens in a short temporal moment (e.g., "guitar solo") [17]. Based on our previous experience [18], [19], sparse coding based audio-word feature significantly outperforms other conventional coding method, such as Vector Quantization. However, one of the main drawback of sparse coding is the expensive computational cost [3].…”

Section: Introductionmentioning

confidence: 99%

“…In our previous audio-word extraction system [18], [19], the computational cost of a given song's encoding is proportional to O(NT ), where N is the number of frames and T is the average time spend on computing sparse coding for each frame. Based on such relation, we could further decompose the computational cost reduction problem into two subproblems: how to reduce N and how to reduce T ?…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Towards a more efficient sparse coding based audio-word feature extraction system

Yeh

Yang

2013

2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference

Self Cite

View full text Add to dashboard Cite

This paper is concerned with the efficiency of sparse coding based audio-word feature extraction system. In particular, we have defined and added the concept of early and late temporal pooling to the classic sparse coding based audio-word feature extraction pipeline, and we have tested them on the genre tags subset of the CAL10k data set. We define temporal pooling as any functions that are able to transforms the input time series representation into a more temporally compact representation. Under this definition, we have examined the following two temporal pooling functions for improving the feature extraction's efficiency, and they are: Early Texture Window Pooling and Multiple Frame Representation. Early texture window pooling tremendously boost the efficiency by compromising the retrieving accuracy, while multiple frame representation slightly improve both the feature extracting efficiency and retrieving accuracy. Overall, our best feature extraction setup achieves 0.202 in mean average precision on the genre tags subset of the CAL10k data set.

show abstract

Affective Music Information Retrieval

Wang

Yang

Wang

2016

Human–Computer Interaction Series

Self Cite

View full text Add to dashboard Cite

Much of the appeal of music lies in its power to convey emotions/moods and to evoke them in listeners. In consequence, the past decade witnessed a growing interest in modeling emotions from musical signals in the music information retrieval (MIR) community. In this article, we present a novel generative approach to music emotion modeling, with a specific focus on the valence-arousal (VA) dimension model of emotion. The presented generative model, called acoustic emotion Gaussians (AEG), better accounts for the subjectivity of emotion perception by the use of probability distributions. Specifically, it learns from the emotion annotations of multiple subjects a Gaussian mixture model in the VA space with prior constraints on the corresponding acoustic features of the training music pieces. Such a computational framework is technically sound, capable of learning in an online fashion, and thus applicable to a variety of applications, including user-independent (general) and user-dependent (personalized) emotion recognition and emotion-based music retrieval. We report evaluations of the aforementioned applications of AEG on a largerscale emotion-annotated corpora, AMG1608, to demonstrate the effectiveness of AEG and to showcase how evaluations are conducted for research on emotion-based MIR. Directions of future work are also discussed.

show abstract

A Systematic Evaluation of the Bag-of-Frames Representation for Music Information Retrieval

Cited by 42 publications

References 36 publications

Automatic Music Emotion Classification Using Artificial Neural Network Based on Vocal and Instrumental Sound Timbres

Automatic Music Emotion Classification Using Artificial Neural Network Based on Vocal and Instrumental Sound Timbres

Towards a more efficient sparse coding based audio-word feature extraction system

Affective Music Information Retrieval

Contact Info

Product

Resources

About