Visual Object Detector for Cow Sound Event Detection

Pandeya, Yagya Raj; Bhattarai, Bhuwan; Lee, Joonwhoan

doi:10.1109/access.2020.3022058

Cited by 21 publications

(11 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning-based approaches, on the other hand, typically involve training a neural network on large amounts of annotated audio data and using the trained network to classify audio signals. SED has been applied in various fields, such as identifying bearing faults in noisy industrial environments using deep neural networks [ 8 ], recognizing the cow sound [ 45 , 46 ], and detecting respiratory diseases in patients’ voices [ 9 , 10 ]. More recently, advanced methods such as self-supervised learning [ 43 , 44 ], which uses large amounts of unlabeled audio data for pre-training a model, and multi-task learning [ 47 ], which involves a joint learning approach for sound event detection and localization, have been utilized to improve SED performance.…”

Section: Related Workmentioning

confidence: 99%

BattleSound: A Game Sound Benchmark for the Sound-Specific Feedback Generation in a Battle Game

Shin

Lee

Jun

et al. 2023

Sensors

View full text Add to dashboard Cite

A haptic sensor coupled to a gamepad or headset is frequently used to enhance the sense of immersion for game players. However, providing haptic feedback for appropriate sound effects involves specialized audio engineering techniques to identify target sounds that vary according to the game. We propose a deep learning-based method for sound event detection (SED) to determine the optimal timing of haptic feedback in extremely noisy environments. To accomplish this, we introduce the BattleSound dataset, which contains a large volume of game sound recordings of game effects and other distracting sounds, including voice chats from a PlayerUnknown’s Battlegrounds (PUBG) game. Given the highly noisy and distracting nature of war-game environments, we set the annotation interval to 0.5 s, which is significantly shorter than the existing benchmarks for SED, to increase the likelihood that the annotated label contains sound from a single source. As a baseline, we adopt mobile-sized deep learning models to perform two tasks: weapon sound event detection (WSED) and voice chat activity detection (VCAD). The accuracy of the models trained on BattleSound was greater than 90% for both tasks; thus, BattleSound enables real-time game sound recognition in noisy environments via deep learning. In addition, we demonstrated that performance degraded significantly when the annotation interval was greater than 0.5 s, indicating that the BattleSound with short annotation intervals is advantageous for SED applications that demand real-time inferences.

show abstract

Section: Related Workmentioning

confidence: 99%

BattleSound: A Game Sound Benchmark for the Sound-Specific Feedback Generation in a Battle Game

Shin

Lee

Jun

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…The raw audio requires pre-processing in a suitable format before it is input into the deep neural networks for the MIR task. We used the log-Mel spectrogram for both classification and semantic segmentation tasks because it is proven and found efficient representation for audio classification [19], emotion recognition [20], and sound event detection [21,22]. The data pre-processing during training and testing time is designed to address the fixed-size input to the neural network and memory issues in processing the long audio sequence.…”

Section: Audio Representationmentioning

confidence: 99%

Tracking the Rhythm: Pansori Rhythm Segmentation and Classification Methods and Datasets

2022

Self Cite

View full text Add to dashboard Cite

This paper presents two methods to understand the rhythmic patterns of the voice in Korean traditional music called Pansori. We used semantic segmentation and classification-based structural analysis methods to segment the seven rhythmic categories of Pansori. We propose two datasets; one is for rhythm classification and one is for segmentation. Two classification and two segmentation neural networks are trained and tested in an end-to-end manner. The standard HR network and DeepLabV3+ network are used for rhythm segmentation. A modified HR network and a novel GlocalMuseNet are used for the classification of music rhythm. The GlocalMuseNet outperforms the HR network for Pansori rhythm classification. A novel segmentation model (a modified HR network) is proposed for Pansori rhythm segmentation. The results show that the DeepLabV3+ network is superior to the HR network. The classifier networks are used for time-varying rhythm classification that behaves as the segmentation using overlapping window frames in a spectral representation of audio. Semantic segmentation using the DeepLabV3+ and the HR network shows better results than the classification-based structural analysis methods used in this work; however, the annotation process is relatively time-consuming and costly.

show abstract

“…Strong labels include onset and offset times in the given audio recordings, but weak labels give only the types of audio events. When incomplete information is given, the task is called weakly supervised classification [11][12][13][14][15][16][17]. Using weakly supervised learning with an incompletely labeled dataset might reduce the costs for dataset construction.…”

Section: Introductionmentioning

confidence: 99%

Weakly Supervised U-Net with Limited Upsampling for Sound Event Detection

2023

View full text Add to dashboard Cite

Sound event detection (SED) is the task of finding the identities of sound events, as well as their onset and offset timings from audio recordings. When complete timing information is not available in the training data, but only the event identities are known, SED should be solved by weakly supervised learning. The conventional U-Net with global weighted rank pooling (GWRP) has shown a decent performance, but extensive computation is demanded. We propose a novel U-Net with limited upsampling (LUU-Net) and global threshold average pooling (GTAP) to reduce the model size, as well as the computational overhead. The expansion along the frequency axis in the U-Net decoder was minimized, so that the output map sizes were reduced by 40% at the convolutional layers and 12.5% at the fully connected layers without SED performance degradation. The experimental results on a mixed dataset of DCASE 2018 Tasks 1 and 2 showed that our limited upsampling U-Net (LUU-Net) with GTAP was about 23% faster in training and achieved 0.644 in audio tagging and 0.531 in weakly supervised SED tasks in terms of F1 scores, while U-Net with GWRP showed 0.629 and 0.492, respectively. The major contribution of the proposed LUU-Net is the reduction in the computation time with the SED performance being maintained or improved. The other proposed method, GTAP, further improved the training time reduction and provides versatility for various audio mixing conditions by adjusting a single hyperparameter.

show abstract

Visual Object Detector for Cow Sound Event Detection

Cited by 21 publications

References 26 publications

BattleSound: A Game Sound Benchmark for the Sound-Specific Feedback Generation in a Battle Game

BattleSound: A Game Sound Benchmark for the Sound-Specific Feedback Generation in a Battle Game

Tracking the Rhythm: Pansori Rhythm Segmentation and Classification Methods and Datasets

Weakly Supervised U-Net with Limited Upsampling for Sound Event Detection

Contact Info

Product

Resources

About