2013 IEEE International Conference on Acoustics, Speech and Signal Processing 2013
DOI: 10.1109/icassp.2013.6637748
|View full text |Cite
|
Sign up to set email alerts
|

Evaluating automatically estimated chord sequences

Abstract: In this paper, we perform an in-depth evaluation of a large number of algorithms for chord estimation that have been submitted to the MIREX competitions in 2010, 2011 and 2012. Therefore we first present a rigorous scheme to describe evaluation methods in a sound, unambiguous way that extends previous work specifically to take into account the large variance in chord estimation vocabularies and to perform evaluations on select sets of chords. Then we take a look at the evaluation metrics used so far and propos… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1
1
1
1

Citation Types

0
11
0

Year Published

2013
2013
2022
2022

Publication Types

Select...
5
3

Relationship

0
8

Authors

Journals

citations
Cited by 19 publications
(11 citation statements)
references
References 6 publications
0
11
0
Order By: Relevance
“…[9]), but this is not the focus of the current paper. Instead, our focus is on the observation that any evaluation metric, including the commonly used NHD, is valuable only if the reference annotations are 'correct.…”
Section: Introductionmentioning
confidence: 90%
See 1 more Smart Citation
“…[9]), but this is not the focus of the current paper. Instead, our focus is on the observation that any evaluation metric, including the commonly used NHD, is valuable only if the reference annotations are 'correct.…”
Section: Introductionmentioning
confidence: 90%
“…Indeed, the likelihood function is given as: (9) If the consensuses are known, the experts' biases can be estimated efficiently using maximum likelihood estimation. On the other hand, given an estimate of the biases , the most probable label for data point can be estimated as (10) The EM algorithm starts with a random initialization of and iterates these two steps until convergence (possibly with several restarts).…”
Section: A Crowd Learningmentioning
confidence: 99%
“…The proposed ACE algorithm has been evaluated using the same method proposed in [14] with the software MusOOEvaluator 4 and the pre-set called Mirex2010, on the same database. The Mirex2010 evaluation metric calculates a segment-based score by considering matches between overlapping segments of the estimated chord sequence and the annotated chord sequence.…”
Section: Discussionmentioning
confidence: 99%
“…According to [50], chord symbol recall (CSR) is a suitable metric to evaluate chord estimation performance. Since 2013, MIREX has used CSR to estimate how well the predicted chords match the ground truth:…”
Section: Corpus and Evaluation Resultsmentioning
confidence: 99%