Sound Event Detection with Sequentially Labelled Data Based on Connectionist Temporal Classification and Unsupervised Clustering

Hou, Yuanbo; Kong, Qiuqiang; Li, Shengchen; Plumbley, Mark D.

doi:10.1109/icassp.2019.8683627

Cited by 19 publications

(22 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The features used in this work are log mel-band energies which were previously adopted in [5,17,20], whereas dilated CRNN is proposed to be used as the classifier. CRNNs will be described in next subsections.…”

Section: Methodsmentioning

confidence: 99%

“…The RNN has been combined with convolutional layers, resulting in a CRNN which achieved This work was supported by the national natural science foundation of China (61771200, 6191101570, 6191101514, and 6191101306) and the project of international science and technology collaboration of Guangdong province (2019A050509001). state-of-the-art results in detecting sound events [5,6,17].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sound Event Detection Via Dilated Convolutional Recurrent Neural Networks

Liu

Drossos

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Convolutional recurrent neural networks (CRNNs) have achieved state-of-the-art performance for sound event detection (SED). In this paper, we propose to use a dilated CRNN, namely a CRNN with a dilated convolutional kernel, as the classifier for the task of SED. We investigate the effectiveness of dilation operations which provide a CRNN with expanded receptive fields to capture long temporal context without increasing the amount of CRNN's parameters. Compared to the classifier of the baseline CRNN, the classifier of the dilated CRNN obtains a maximum increase of 1.9%, 6.3% and 2.5% at F1 score and a maximum decrease of 1.7%, 4.1% and 3.9% at error rate (ER), on the publicly available audio corpora of the TUT-SED Synthetic 2016, the TUT Sound Event 2016 and the TUT Sound Event 2017, respectively.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Sound Event Detection Via Dilated Convolutional Recurrent Neural Networks

Liu

Drossos

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…In order to tackle the labeling efforts needed for strong labeling, Hou et al [25] and Wang and Metze [3] proposed the use of sequential labels. The idea of sequential labeling is that rather than annotating the start and end of an event, sequential labeling labels the sequence of event occurrence.…”

Section: Related Workmentioning

confidence: 99%

Semi-Supervised NMF-CNN for Sound Event Detection

Chan

Chin

2021

IEEE Access

View full text Add to dashboard Cite

The lack of strongly labeled data can limit the potential of a Sound Event Detection (SED) system trained using deep learning approaches. To address this issue, this paper proposes a novel method to approximate strong labels for the weakly labeled data using Nonnegative Matrix Factorization (NMF) in a supervised manner. Using a combinative transfer learning and semi-supervised learning framework, two different Convolutional Neural Networks (CNN) are trained using synthetic data, approximated strongly labeled data, and unlabeled data where one model will produce the audio tags. In contrast, the other will produce the frame-level prediction. The proposed methodology is then evaluated on three different subsets of the Detection and Classification of Acoustic Scenes and Events (DCASE) 2020 dataset: validation dataset, challenge evaluation dataset, and public YouTube evaluation dataset. Based on the results, our proposed methodology outperforms the baseline system by a minimum of 7% across these three different data subsets. In addition, our proposed method also outperforms the top 3 submissions from the DCASE 2019 challenge task 4 on the validation and public YouTube evaluation datasets. Our system performance is also competitive against the top submission in DCASE 2020 challenge task 4 on the challenge evaluation data. A post-challenge analysis was also performed using the validation dataset, which revealed the causes of the performance difference between our system and the top submission of the DCASE 2020 challenge task 4. The leading causes that we observed are 1) detection threshold tuning method and 2) augmentation techniques used. We observed that our system could perform better than the first place submission by 1.5% by changing our detection threshold tuning method. In addition, the post-challenge analysis also revealed that our system is more robust than the top submission in DCASE 2020 challenge task 4 on long-duration audio clips, where we outperformed them by 37%.

show abstract

“…Convolutional layers with gated linear units (GLUs) [15] are applied to learn local shift-invariant patterns from the acoustic feature. By using GLUs, the model can learn to attend to target events and ignore unrelated sounds [16]. The pooling operation is implemented by convolution with strides (1, 2), which means the stride of the convolution along the time is 1 to preserve the time resolution of the input.…”

Section: The Audio Branchmentioning

confidence: 99%

“…For evaluation metrics, event-based precision (P ), recall (R ), F-score and Error rate (ER ) [21] are used. Compared with segment-based metrics used in previous studies [22,16,2], event-based metrics are more rigorous and accurate to measure the location of events. Higher P, R, F and lower ER indicate a better performance.…”

Section: Dataset Baseline and Experiments Setupmentioning

confidence: 99%

Rule-Embedded Network for Audio-Visual Voice Activity Detection in Live Musical Video Streams

Hou

Deng

Zhu³

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Detecting anchor's voice in live musical streams is an important preprocessing step for music and speech signal processing. Existing approaches to voice activity detection (VAD) primarily rely on audio, however, audio-based VAD is difficult to effectively focus on the target voice in noisy environments. This paper proposes a rule-embedded network to fuse the audio-visual (A-V) inputs for better detection of the target voice. The core role of the rule in the model is to coordinate the relation between the bi-modal information and use visual representations as a mask to filter out the information of non-target sound. Experiments show that: 1) with the help of cross-modal fusion using the proposed rule, the detection results of the A-V branch outperform that of the audio branch in the same model framework; 2) the performance of the bimodal A-V model far outperforms that of audio-only models, indicating that the incorporation of both audio and visual signals is highly beneficial for VAD. To attract more attention to the cross-modal music and audio signal processing, a new live musical video corpus with frame-level labels is introduced.

show abstract

Sound Event Detection with Sequentially Labelled Data Based on Connectionist Temporal Classification and Unsupervised Clustering

Cited by 19 publications

References 16 publications

Sound Event Detection Via Dilated Convolutional Recurrent Neural Networks

Sound Event Detection Via Dilated Convolutional Recurrent Neural Networks

Semi-Supervised NMF-CNN for Sound Event Detection

Rule-Embedded Network for Audio-Visual Voice Activity Detection in Live Musical Video Streams

Contact Info

Product

Resources

About