Deep Learning for Orca Call Type Identification — A Fully Unsupervised Approach

Bergler, Christian; Schmitt, Manuel; Cheng, Rachael Xi; Maier, Andreas; Barth, Volker; Nöth, Elmar

doi:10.21437/interspeech.2019-1857

Cited by 4 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An opportunity to avoid having to manually choose the right set of features and tune their settings is to use features learnt using deep representation learning (as opposed to handcrafted features like PAFs). Following this approach, auto-encoder artificial neural networks have been used by Goffinet et al [28] on mice and zebra finch vocalisations, by Bergler et al [29] to cluster orca calls, by Rowe et al [30] to cluster bird vocalisations by species, and by Tolkova et al [31] to discriminate between background noise and bird vocalisations.…”

Section: Vocalisations Feature Extraction and Clusteringmentioning

confidence: 99%

Deep audio embeddings for vocalisation clustering

et al. 2023

View full text Add to dashboard Cite

The study of non-human animals’ communication systems generally relies on the transcription of vocal sequences using a finite set of discrete units. This set is referred to as a vocal repertoire, which is specific to a species or a sub-group of a species. When conducted by human experts, the formal description of vocal repertoires can be laborious and/or biased. This motivates computerised assistance for this procedure, for which machine learning algorithms represent a good opportunity. Unsupervised clustering algorithms are suited for grouping close points together, provided a relevant representation. This paper therefore studies a new method for encoding vocalisations, allowing for automatic clustering to alleviate vocal repertoire characterisation. Borrowing from deep representation learning, we use a convolutional auto-encoder network to learn an abstract representation of vocalisations. We report on the quality of the learnt representation, as well as of state of the art methods, by quantifying their agreement with expert labelled vocalisation types from 8 datasets of other studies across 6 species (birds and marine mammals). With this benchmark, we demonstrate that using auto-encoders improves the relevance of vocalisation representation which serves repertoire characterisation using a very limited number of settings. We also publish a Python package for the bioacoustic community to train their own vocalisation auto-encoders or use a pretrained encoder to browse vocal repertoires and ease unit wise annotation.

show abstract

Section: Vocalisations Feature Extraction and Clusteringmentioning

confidence: 99%

Deep audio embeddings for vocalisation clustering

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Contrary to supervised classifier neural networks for which the bioacoustics community often uses off-the-shelf architecture [40], there is to our knowledge not yet an architecture of choice for auto-encoder networks. Bergler et al [29] use a ResNet-18 architecture, and Goffinet et al [28] use a custom architecture consisting in successive blocks of convolution, batch normalisation and rectifier linear units (ReLU). For the encoder, these successive blocks gradually increase the number of feature maps while decreasing the spectro-temporal dimensions.…”

Section: Auto-encoder Network Architecturementioning

confidence: 99%

“…An opportunity to avoid having to manually choose the right set of features and tune their settings is to use features learnt using deep representation learning (as opposed to handcrafted features like PAFs). Following this approach, auto-encoder artificial neural networks have been used by Goffinet et al [28] on mice and zebra finch vocalisations, by Bergler et al [29] to cluster orca calls and by Tolkova et al [30] to discriminate between background noise and bird vocalisations.…”

Section: Vocalisations Feature Extraction and Clusteringmentioning

confidence: 99%

“…Auto-encoders trained on vocalisation spectrograms have recently shown promising results [28,29,32], we thus compare the embedding space they yield to that of more traditional methods, especially how it allows to cluster units by acoustic similarity.…”

Section: Vocalisations Feature Extraction and Clusteringmentioning

confidence: 99%

“…To follow up on this research, we propose studying vocalisation clustering, especially the choice of audio representation and its effect on the subsequent clustering. Auto-encoders trained on vocalisation spectrograms have recently shown promising results [28, 29, 32], we thus compare the embedding space they yield to that of more traditional methods, especially how it allows to cluster units by acoustic similarity. Taking detected vocalisations as an input, clusters of similar vocalisations can thus be presented to an operator, facilitating vocal repertoire characterisation as well as annotation (e.g.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep audio embeddings for vocalisation clustering

Best

Paris

Glotin

et al. 2023

Preprint

View full text Add to dashboard Cite

The study of non-human animals' communication systems generally relies on the transcription of vocal sequences using a finite set of discrete units. This set is referred to as a vocal repertoire, which is specific to a species or a sub-group of a species. When conducted by human experts, the formal description of vocal repertoires can be laborious and / or biased. This motivates a computerised assistance for this procedure, for which machine learning algorithms represent a good opportunity. Unsupervised clustering algorithms are suited for grouping close points together, provided a relevant representation. This paper therefore studies a new method for encoding vocalisations, allowing for automatic clustering to alleviate vocal repertoire characterisation. Borrowing from deep representation learning, we use a convolutional auto-encoder network to learn an abstract representation of vocalisations. We report on the quality of the learnt representation, as well as of state of the art methods, by quantifying their agreement with expert labelled vocalisation types from 7 datasets of other studies across 6 species (birds and marine mammals). With this benchmark, we demonstrate that using auto-encoders improves the relevance of vocalisation representation which serves repertoire characterisation, and so using a very limited amount of settings. We also publish a Python package for the bioacoustic community to train their own vocalisation auto-encoders or use a pretrained encoder to browse vocal repertoires and ease unit wise annotation.

show abstract

ORCA-SPY enables killer whale sound source simulation, detection, classification and localization using an integrated deep learning-based segmentation

Hauer,

Nöth,

Barnhill

et al. 2023

Sci Rep

Self Cite

View full text Add to dashboard Cite

Acoustic identification of vocalizing individuals opens up new and deeper insights into animal communications, such as individual-/group-specific dialects, turn-taking events, and dialogs. However, establishing an association between an individual animal and its emitted signal is usually non-trivial, especially for animals underwater. Consequently, a collection of marine species-, array-, and position-specific ground truth localization data is extremely challenging, which strongly limits possibilities to evaluate localization methods beforehand or at all. This study presents ORCA-SPY, a fully-automated sound source simulation, classification and localization framework for passive killer whale (Orcinus orca) acoustic monitoring that is embedded into PAMGuard, a widely used bioacoustic software toolkit. ORCA-SPY enables array- and position-specific multichannel audio stream generation to simulate real-world ground truth killer whale localization data and provides a hybrid sound source identification approach integrating ANIMAL-SPOT, a state-of-the-art deep learning-based orca detection network, followed by downstream Time-Difference-Of-Arrival localization. ORCA-SPY was evaluated on simulated multichannel underwater audio streams including various killer whale vocalization events within a large-scale experimental setup benefiting from previous real-world fieldwork experience. Across all 58,320 embedded vocalizing killer whale events, subject to various hydrophone array geometries, call types, distances, and noise conditions responsible for a signal-to-noise ratio varying from $$-14.2$$ - 14.2 dB to 3 dB, a detection rate of 94.0 % was achieved with an average localization error of 7.01$$^\circ$$ ∘ . ORCA-SPY was field-tested on Lake Stechlin in Brandenburg Germany under laboratory conditions with a focus on localization. During the field test, 3889 localization events were observed with an average error of 29.19$$^\circ$$ ∘ and a median error of 17.54$$^\circ$$ ∘ . ORCA-SPY was deployed successfully during the DeepAL fieldwork 2022 expedition (DLFW22) in Northern British Columbia, with a mean average error of 20.01$$^\circ$$ ∘ and a median error of 11.01$$^\circ$$ ∘ across 503 localization events. ORCA-SPY is an open-source and publicly available software framework, which can be adapted to various recording conditions as well as animal species.

show abstract

Deep Learning for Orca Call Type Identification — A Fully Unsupervised Approach

Cited by 4 publications

References 0 publications

Deep audio embeddings for vocalisation clustering

Deep audio embeddings for vocalisation clustering

Deep audio embeddings for vocalisation clustering

ORCA-SPY enables killer whale sound source simulation, detection, classification and localization using an integrated deep learning-based segmentation

Contact Info

Product

Resources

About