Bioacoustic Event Detection with Self-Supervised Contrastive Learning

Abstract

While deep learning has revolutionized ecological data analysis, existing strategies often rely on supervised learning, which is subject to limitations on real-world applicability. In this paper, we apply self supervised deep learning methods to bioacoustic data to enable unsupervised detection of bioacoustic event boundaries. We propose a convolutional deep neural network that operates on the raw waveform directly and is trained in accordance with the Noise Contrastive Estimation principle, which enables the system to detect spectral changes in the input acoustic stream. The model learns a representation of the input audio sampled at low frequency that encodes information regarding dissimilarity between sequential acoustic windows. During inference, we use a peak finding algorithm to search for regions of high dissimilarity in order to identify temporal boundaries of bioacoustic events. We report results using these techniques to detect sperm whale (Physeter macrocephalus) coda clicks in real-world recordings, and we demonstrate the viability of analyzing the vocalizations of other species (e.g. Bengalese finch syllable segmentation) in addition to other data modalities (e.g. animal behavioral dynamics, embryo development and tracking). We find that the self-supervised deep representation learning-based technique outperforms established threshold-based baseline methods without requiring manual annotation of acoustic datasets. Quantitatively, our approach yields a maximal R-value and F1-score of 0.887 and 0.876, respectively, and an area under the Precision-Recall curve (PR-AUC) of 0.917, while a threshold detector based on signal energy amplitude returns a maximal R-value and F1-score of 0.620 and 0.576, respectively, and a PR-AUC of 0.571. The findings of this paper establish the validity of unsupervised bioacoustic event detection using deep neural networks and self-supervised contrastive learning as an effective alternative to conventional techniques that leverage supervised methods for signal presence indication. Providing a means for highly accurate unsupervised detection, this paper serves as an important step towards developing a fully automated system for real-time acoustic monitoring of bioacoustic signals in real-world acoustic data. All code and data used in this study are available online.

Elephants and algorithms: a review of the current and future role of AI in elephant monitoring

“…This area of research is currently experiencing significant activity. Notable examples include self-supervision in images to learn valuable information from the images and videos automatically [180,181], and using unsupervised methods to improve and automate bio-acoustics analysis of PAM datasets without labelling [139,140]. Moreover, innovative techniques have emerged for animal behaviour monitoring without the need for labels using unsupervised techniques [91,103].…”

“…Deep learning source separation techniques can filter the data to provide separate audio streams for calls from different sources, showing promise across various bio-acoustic datasets [138]. Unsupervised source separation techniques have also demonstrated success in classification improvement [139], and unsupervised techniques have also been used to improve event detection [140]. The benefits of unsupervised learning are discussed more in §6.2.…”