Learning Deep Models from Synthetic Data for Extracting Dolphin Whistle Contours

Abstract

We present a learning-based method for extracting whistles of toothed whales (Odontoceti) in hydrophone recordings. Our method represents audio signals as time-frequency spectrograms and decomposes each spectrogram into a set of time-frequency patches. A deep neural network learns archetypical patterns (e.g., crossings, frequency modulated sweeps) from the spectrogram patches and predicts time-frequency peaks that are associated with whistles. We also developed a comprehensive method to synthesize training samples from background environments and train the network with minimal human annotation effort. We applied the proposed learn-from-synthesis method to a subset of the public Detection, Classification, Localization, and Density Estimation (DCLDE) 2011 workshop data to extract whistle confidence maps, which we then processed with an existing contour extractor to produce whistle annotations. The F1-score of our best synthesis method was 0.158 greater than our baseline whistle extraction algorithm (~25% improvement) when applied to common dolphin (Delphinus spp.) and bottlenose dolphin (Tursiops truncatus) whistles.

Behavioural Responses of Common Dolphins Delphinus delphis to a Bio-Inspired Acoustic Device for Limiting Fishery By-Catch

“…This would involve an accurate identification of each whistle. However, this task is complex, and detection techniques developed so far have not been able to solve this problem without errors [69][70][71][72][73][74][75][76][77][78][79][80][81][82]84,85]. If whistles are first identified, then a classification of whistles using unsupervised classification techniques such as UMAP [67] could be applied, also using a metric to determine the optimal number of clusters [95].…”

“…Thus, to detect and identify whistles, a vocalisation tracking method is necessary. Selecting vocalisations in an audio track is a complex task, and several techniques have already been developed to achieve it: statistical modelling of whistles [69][70][71][72][73][74], tracking algorithms based on hand-picked parameters [75][76][77], image processing approaches [78][79][80][81][82][83] or deep learning models associated with clustering methods [84,85]. For our dataset, we chose to adapt a tracking algorithm developed during the DECAV project [86].…”

Tracking time differences of arrivals of multiple sound sources in the presence of clutter and missed detections

“…In the future, larger training datasets should be considered, either by the hand annotation of the field data or, potentially, by constructing them from existing data with the help of the deep learning procedures. 56 The parameters reported in this paper were trained based on the whistle cross-correlograms and used for tracking all of the measurement types, including echolocation clicks and combined clicks and whistles. The parameters related to the evolution of the TDOA tracks are expected to be similar between the whistle or click cross-correlograms.…”

Automated detection of dolphin whistles with convolutional networks and transfer learning

“…In particular, Convolutional Neural Networks (CNN) (LeCun et al, 1995 ) have been applied to detection of whales vocalizations, producing false-positive rates that are orders of magnitude lower than traditional algorithms, while substantially increasing the ability to detect calls (Jiang et al, 2019 ; Shiu et al, 2020 ). Deep learning has also been used to automatically classify dolphin whistles into specific categories (Li et al, 2021 ) and to extract whistle contours by exploiting peak tracking algorithms (Li et al, 2020 ) or by training CNN-based semantic segmentation models (Jin et al, 2022 ).…”