Segmentation, Classification, and Visualization of Orca Calls Using Deep Learning

Schröter, Hendrik; Nöth, Elmar; Maier, Andreas; Cheng, Rachael Xi; Barth, Volker; Bergler, Christian

doi:10.1109/icassp.2019.8683785

Cited by 10 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Potentially, it is possible to translate that stream of voltage fluctuations into signs, which could be interpreted by humans using approaches from deep learning based natural language processing such as machine translation, or even from animal linguistics, where e.g. killer whale calls are identified, segmented, extracted and classified from ongoing continuous sound streams according to recurring feature patterns [87][88][89][90]. By that, we are convinced that our approach might further push the progress in neuroscience in order to extract meaningful information from continuous electrophysiological data streams.…”

Section: Discussionmentioning

confidence: 99%

Deep learning based decoding of local field potential events

Schilling

Gerum

Boehm

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

How is information processed in the cerebral cortex? To answer this question a lot of effort has been undertaken to create novel and to further develop existing neuroimaging techniques. Thus, a high spatial resolution of fMRI devices was the key to exactly localize cognitive processes. Furthermore, an increase in time-resolution and number of recording channels of electro-physiological setups has opened the door to investigate the exact timing of neural activity. However, in most cases the recorded signal is averaged over many (stimulus) repetitions, which erases the fine-structure of the neural signal. Here, we show that an unsupervised machine learning approach can be used to extract meaningful information from electro-physiological recordings on a single-trial base. We use an auto-encoder network to reduce the dimensions of single local field potential (LFP) events to create interpretable clusters of different neural activity patterns. Strikingly, certain LFP shapes correspond to latency differences in different recording channels. Hence, LFP shapes can be used to determine the direction of information flux in the cerebral cortex. Furthermore, after clustering, we decoded the cluster centroids to reverse-engineer the underlying prototypical LFP event shapes. To evaluate our approach, we applied it to both neural extra-cellular recordings in rodents, and intra-cranial EEG recordings in humans. Finally, we find that single channel LFP event shapes during spontaneous activity sample from the realm of possible stimulus evoked event shapes. A finding which so far has only been demonstrated for multi-channel population coding.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep learning based decoding of local field potential events

Schilling

Gerum

Boehm

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Motivated by the need for better conservation tools, researchers have begun applying speech recognition algorithms to animal vocalizations [ 14 ]. Schröter et al constructed a new model to recognize different types of killer whale calls [ 15 ]. The accuracy can reach 97%, and successfully visualized the feature map.…”

Section: Introductionmentioning

confidence: 99%

Automatic Recognition of Giant Panda Attributes from Their Vocalizations Based on Squeeze-and-Excitation Network

Zhao

Zhang

Hou

et al. 2022

Sensors

View full text Add to dashboard Cite

The giant panda (Ailuropoda melanoleuca) has long attracted the attention of conservationists as a flagship and umbrella species. Collecting attribute information on the age structure and sex ratio of the wild giant panda populations can support our understanding of their status and the design of more effective conservation schemes. In view of the shortcomings of traditional methods, which cannot automatically recognize the age and sex of giant pandas, we designed a SENet (Squeeze-and-Excitation Network)-based model to automatically recognize the attributes of giant pandas from their vocalizations. We focused on the recognition of age groups (juvenile and adult) and sex of giant pandas. The reason for using vocalizations is that among the modes of animal communication, sound has the advantages of long transmission distances, strong penetrating power, and rich information. We collected a dataset of calls from 28 captive giant panda individuals, with a total duration of 1298.02 s of recordings. We used MFCC (Mel-frequency Cepstral Coefficients), which is an acoustic feature, as inputs for the SENet. Considering that small datasets are not conducive to convergence in the training process, we increased the size of the training data via SpecAugment. In addition, we used focal loss to reduce the impact of data imbalance. Our results showed that the F1 scores of our method for recognizing age group and sex reached 96.46% ± 5.71% and 85.85% ± 7.99%, respectively, demonstrating that the automatic recognition of giant panda attributes based on their vocalizations is feasible and effective. This more convenient, quick, timesaving, and laborsaving attribute recognition method can be used in the investigation of wild giant pandas in the future.

show abstract

“…In order to use these methods, the one-dimensional audio signal must first be converted to a two-dimensional image-like representation. The most common audio visualization is the spectrogram, which has been used as the input to a variety of audio classification tasks [16] , [41] , [42] , [43] , [44] , [45] , [46] . In this paper, we focus on the application of transfer learning to classify respiratory sound characteristics, building on our previous work [16] .…”

Section: Introductionmentioning

confidence: 99%

Evaluation of Respiratory Sounds Using Image-Based Approaches for Health Measurement Applications

Cohen-McFarlane

Wallace

et al. 2022

IEEE Open J. Eng. Med. Biol.

View full text Add to dashboard Cite

Goal: The evaluation of respiratory events using audio sensing in an at-home setting can be indicative of worsening health conditions. This paper investigates the use of image-based transfer learning applied to five audio visualizations to evaluate three classification tasks (C1: wet vs. dry vs. whooping cough vs. restricted breathing; C2: wet vs. dry cough; C3: cough vs. restricted breathing). Methods: The five visualizations (linear spectrogram, logarithmic spectrogram, Mel-spectrogram, wavelet scalograms, and aggregate images) are applied to a pre-trained AlexNet image classifier for all tasks. Results: The aggregate image-based classifier achieved the highest overall performance across all tasks with C1, C2, and C3 having testing accuracies of 0.88, 0.88, and 0.91 respectively. However, the Mel-spectrogram method had the highest testing accuracy (0.94) for C2. Conclusions: The classification of respiratory events using aggregate image inputs to transfer learning approaches may help healthcare professionals by providing information that would otherwise be unavailable to them.

show abstract

Segmentation, Classification, and Visualization of Orca Calls Using Deep Learning

Cited by 10 publications

References 7 publications

Deep learning based decoding of local field potential events

Deep learning based decoding of local field potential events

Automatic Recognition of Giant Panda Attributes from Their Vocalizations Based on Squeeze-and-Excitation Network

Evaluation of Respiratory Sounds Using Image-Based Approaches for Health Measurement Applications

Contact Info

Product

Resources

About