A Survey of Evaluation in Music Genre Recognition

Sturm, Bob L.

doi:10.1007/978-3-319-12093-5_2

Cited by 72 publications

(96 citation statements)

References 254 publications

(149 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This article rigorously addresses all of these, significantly extending our analysis of GTZAN in several practical ways. Our work not only illuminates results reported by a significant amount of work, but also provides a critical piece to address the non-trivial problems associated with evaluating music machine listening systems in valid ways [1,83,84,[103][104][105].…”

Section: Introductionsupporting

confidence: 53%

“…This article is not concerned with other public datasets, which may or may not suffer from the same faults as GTZAN, but which have certainly been used in fewer published works than GTZAN [1]. This article is not concerned with the validity, well-posedness, value, usefulness, or applicability of MGR; or whether MGR is "replaced by," or used in the service of, e.g., music similarity, autotagging, or the like.…”

Section: Delimitationsmentioning

confidence: 99%

“…This article is not concerned with the validity, well-posedness, value, usefulness, or applicability of MGR; or whether MGR is "replaced by," or used in the service of, e.g., music similarity, autotagging, or the like. It is not concerned with making general conclusions about or criticisms of MGR or evaluation in MGR, which are comprehensively addressed in other works, e.g., [1,83,84,[105][106][107][108][109]. Finally, it is not concerned with how, or even whether it is possible, to create faultless datasets for MGR, music similarity, autotagging, and the like.…”

Section: Delimitationsmentioning

confidence: 99%

“…First, its usefulness has been challenged: "recent publications are no longer using GTZAN"; or, "it is already a commonly accepted opinion that the GTZAN dataset should be discarded"; or, "better datasets are available." The fact is, of all datasets that are publicly available, GTZAN is the one appearing most in published MGR work [1]. The fact is, more papers use GTZAN in the past three years than in the first eight years of its existence (Fig.…”

Section: Criticismsmentioning

confidence: 99%

“…Our recent review of over 467 published works in music genre recognition (MGR) [1] shows that the most-used public dataset is GTZAN, 1 appearing in the evaluations of 100 works . GTZAN is composed of 1,000 half-minute music audio excerpts singly labeled in ten categories [92,93]; and though its use is so widespread, it has always been missing metadata identifying its contents.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

The State of the Art Ten Years After a State of the Art: Future Research in Music Information Retrieval

Sturm

2014

Journal of New Music Research

Self Cite

View full text Add to dashboard Cite

The GTZAN dataset appears in at least 100 published works, and is the most-used public dataset for evaluation in machine listening research for music genre recognition (MGR). Our recent work, however, shows GTZAN has several faults (repetitions, mislabelings, and distortions), which challenge the interpretability of any result derived using it. In this article, we disprove the claims that all MGR systems are affected in the same ways by these faults, and that the performances of MGR systems in GTZAN are still meaningfully comparable since they all face the same faults. We identify and analyze the contents of GTZAN, and provide a catalog of its faults. We review how GTZAN has been used in MGR research, and find few indications that its faults have been known and considered. Finally, we rigorously study the effects of its faults on evaluating five different MGR systems. The lesson is not to banish GTZAN, but to use it with consideration of its contents.

show abstract

Section: Introductionsupporting

confidence: 53%

Section: Delimitationsmentioning

confidence: 99%

Section: Delimitationsmentioning

confidence: 99%

Section: Criticismsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

The State of the Art Ten Years After a State of the Art: Future Research in Music Information Retrieval

Sturm

2014

Journal of New Music Research

Self Cite

View full text Add to dashboard Cite

show abstract

Correspondence of categorical and feature‐based representations of music in the human brain

2020

View full text Add to dashboard Cite

Introduction Humans tend to categorize auditory stimuli into discrete classes, such as animal species, language, musical instrument, and music genre. Of these, music genre is a frequently used dimension of human music preference and is determined based on the categorization of complex auditory stimuli. Neuroimaging studies have reported that the superior temporal gyrus (STG) is involved in response to general music‐related features. However, there is considerable uncertainty over how discrete music categories are represented in the brain and which acoustic features are more suited for explaining such representations. Methods We used a total of 540 music clips to examine comprehensive cortical representations and the functional organization of music genre categories. For this purpose, we applied a voxel‐wise modeling approach to music‐evoked brain activity measured using functional magnetic resonance imaging. In addition, we introduced a novel technique for feature‐brain similarity analysis and assessed how discrete music categories are represented based on the cortical response pattern to acoustic features. Results Our findings indicated distinct cortical organizations for different music genres in the bilateral STG, and they revealed representational relationships between different music genres. On comparing different acoustic feature models, we found that these representations of music genres could be explained largely by a biologically plausible spectro‐temporal modulation‐transfer function model. Conclusion Our findings have elucidated the quantitative representation of music genres in the human cortex, indicating the possibility of modeling this categorization of complex auditory stimuli based on brain activity.

show abstract

Finding Music in Music Data: A Summary of the DaCaRyH Project

Ben-Tal¹,

Sturm

Quinton

et al. 2019

Current Research in Systematic Musicology

Self Cite

View full text Add to dashboard Cite

The international research project, "Data science for the study of calypsorhythm through history" (DaCaRyH), involved a collaboration between ethnomusicologists, computer scientists, and a composer. The primary aim of DaCaRyH was to explore how ethnomusicology could inform data science, and vice versa. Its secondary aim focused on creative applications of the results. This article summarises the results of the project, and more broadly discusses the benefits and challenges in such interdisciplinary research. It concludes with suggestions for reducing the barriers to similar work.

show abstract

A Survey of Evaluation in Music Genre Recognition

Cited by 72 publications

References 254 publications

The State of the Art Ten Years After a State of the Art: Future Research in Music Information Retrieval

The State of the Art Ten Years After a State of the Art: Future Research in Music Information Retrieval

Correspondence of categorical and feature‐based representations of music in the human brain

Finding Music in Music Data: A Summary of the DaCaRyH Project

Contact Info

Product

Resources

About