A Novel 1D State Space for Efficient Music Rhythmic Analysis

Heydari, Mojtaba; McCallum, Matthew; Ehmann, Andreas F.; Duan, Zhiyao

doi:10.1109/icassp43922.2022.9747557

“…Many of the proposed music rhythmic analysis approaches [1,5,7] perform in an offline fashion while others [6,9,10] are capable of extracting music rhythmic features causally and even in real-time.…”

Section: Introductionmentioning

confidence: 99%

Singing Beat Tracking With Self-supervised Front-end and Linear Transformers

Heydari¹,

Duan²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

<p>Tracking beats of singing voices without the presence of musical accompaniment can find many applications in music production, automatic song arrangement, and social media interaction. Its main challenge is the lack of strong rhythmic and harmonic patterns that are important for music rhythmic analysis in general. Even for human listeners, this can be a challenging task. As a result, existing music beat tracking systems fail to deliver satisfactory performance on singing voices. In this paper, we propose singing beat tracking as a novel task, and propose the first approach to solving this task. Our approach leverages semantic information of singing voices by employing pre-trained self-supervised WavLM and DistilHuBERT speech representations as the front-end and uses a self-attention encoder layer to predict beats. To train and test the system, we obtain separated singing voices and their beat annotations using source separation and beat tracking on complete songs, followed by manual corrections. Experiments on the 741 separated vocal tracks of the GTZAN dataset show that the proposed system outperforms several state-of-the-art music beat tracking methods by a large margin in terms of beat tracking accuracy. Ablation studies also confirm the advantages of pre-trained self-supervised speech representations over generic spectral features.</p>

show abstract

“…Many of the proposed music rhythmic analysis approaches [1,5,7] perform in an offline fashion while others [6,9,10] are capable of extracting music rhythmic features causally and even in real-time.…”

Section: Introductionmentioning

confidence: 99%

Singing Beat Tracking With Self-supervised Front-end and Linear Transformers

Heydari¹,

Duan²

2022

Preprint

Self Cite

0

View full text Add to dashboard Cite

<p>Tracking beats of singing voices without the presence of musical accompaniment can find many applications in music production, automatic song arrangement, and social media interaction. Its main challenge is the lack of strong rhythmic and harmonic patterns that are important for music rhythmic analysis in general. Even for human listeners, this can be a challenging task. As a result, existing music beat tracking systems fail to deliver satisfactory performance on singing voices. In this paper, we propose singing beat tracking as a novel task, and propose the first approach to solving this task. Our approach leverages semantic information of singing voices by employing pre-trained self-supervised WavLM and DistilHuBERT speech representations as the front-end and uses a self-attention encoder layer to predict beats. To train and test the system, we obtain separated singing voices and their beat annotations using source separation and beat tracking on complete songs, followed by manual corrections. Experiments on the 741 separated vocal tracks of the GTZAN dataset show that the proposed system outperforms several state-of-the-art music beat tracking methods by a large margin in terms of beat tracking accuracy. Ablation studies also confirm the advantages of pre-trained self-supervised speech representations over generic spectral features.</p>

show abstract