Agreement among human and automated transcriptions of global songs

Abstract

Cross-cultural musical analysis requires standardized symbolic representation of sounds such as score notation. However, transcription into notation is usually conducted manually by ear, which is time-consuming and subjective. Our aim is to evaluate the reliability of existing methods for transcribing songs from diverse societies. We had 3 experts independently transcribe a sample of 32 excerpts of traditional monophonic songs from around the world (half a cappella, half with instrumental accompaniment). 16 songs also had pre-existing transcriptions created by 3 different experts. We compared these human transcriptions against one another and against 10 automatic music transcription algorithms. We found that human transcriptions can be sufficiently reliable (~90% agreement, κ ~.7), but current automated methods are not (<60% agreement, κ <.4). No automated method clearly outperformed others, in contrast to our predictions. These results suggest that improving automated methods for cross-cultural music transcription is critical for diversifying MIR.

Cultural evolution and music

“…In conclusion, we feel that researchers have barely scratched the surface of the available datasets and methodologies. For example, there is a rich literature on music information retrieval, including automated methods for extracting pitch and melody (Benetos et al, 2019;Ozaki et al, 2021), that has yet to be fully leveraged for studies of cultural evolution. As researchers continue to collaborate across disciplinary boundaries and take advantage of the diversity of perspectives and approaches from di erent communities (Jacoby et al, 2020), we think that music has the potential to become a powerful research model for cultural evolution.…”

Vocal imprecision as a universal constraint on the structure of musical scales

“…The overall approach of the present study is to move beyond the study of abstract instrumental tunings in art music traditions and investigate the actual production of melodies in indigenous and traditional vocal music, and to do so at a global level that accounts for the diverse panoply of scale types and singing styles worldwide. Some recent comparative musicology studies have analyzed cross-cultural features of melodies 18,19 and scales 20,21 , and some have looked at speci c pitch-class features, including how these features relate to scale structure, although within a relatively small sample [22][23][24][25] . However, there has yet to be a globallycomprehensive, computational survey of the structural features of pitch-classes using a worldwide database of recorded vocal melodies.…”

The Exceptions and the Rules in global musical diversity

“…In this paper we use a measure of relative feature frequency, which we call unusualness, to account for cross-cultural variation in frequency of musical features, and capture musical diversity in a framework designed for cross-cultural comparison Unusualness as a concept can be defined in many, often Eurocentric ways but has rarely been measured quantitatively (Savage, 2022). One study by Panteli, Benetos and Dixon (2017) used Music Information Retrieval (MIR) to quantitatively identify musical outliers in a global sample, but because they did not have any ground-truth human annotated data on musical features, their results are difficult to interpret from a musicological perspective (Ozaki et al, 2021). Utilising the cross-cultural framework and human "Cantometric" annotated data for a large number of songs allows us to calculate unusualness from the relative regional frequency of their annotations, also known as the log-likelihood (Figure 1).…”