Directional embedding based semi-supervised framework for bird vocalization segmentation

Abstract

This paper proposes a data-efficient, semi-supervised, two-pass framework for segmenting bird vocalizations. The framework utilizes a binary classification model to categorize frames of an input audio recording into the background or bird vocalization. The first pass of the framework automatically generates training labels from the input recording itself, while model training and classification is done during the second pass. The proposed framework utilizes a reference directional model for obtaining a feature representation called directional embeddings (DE). This reference directional model acts as an acoustic model for bird vocalizations and is obtained using the mixtures of Von-Mises Fisher distribution (moVMF). The proposed DE space only contains information about bird vocalizations, while no information about the background disturbances is reflected. The framework employs supervised information only for obtaining the reference directional model and avoids the background modeling. Hence, it can be regarded as semi-supervised in nature. The proposed framework is tested on approximately 79000 vocalizations of seven different bird species. The performance of the framework is also analyzed in the presence of noise at different SNRs. Experimental results convey that the proposed framework performs better than the existing bird vocalization segmentation methods.

Systematic review of machine learning methods applied to ecoacoustics and soundscape monitoring

“…We found studies that use datasets from 24 countries. The country with the most publications referring to datasets collected in its territory is Australia with 16 studies [58] , [59] , [31] , [26] , [60] , [61] , [62] , [63] , [64] , [50] , [65] , [66] , [34] , [43] , [67] , [9] ; followed by the United States with 14 publications [68] , [48] , [69] , [70] , [71] , [10] , [72] , [35] , [73] , [74] , [55] , [75] , [76] , [23] ; Brazil with 13 publications [40] , [27] , [60] , [11] , [77] , [28] , [78] , [79] , [80] , [57] , [46] , [47] , [81] ; Costa Rica [12] , [40] , [25] , [45] , Malaysia [42] , [82] , [83] , [84] and the United Kingdom [22] , [85] , [32] , [86] with four publications each; Colombia [87] , [8] , [88] and France [85] , [89] , …”

“…Spectral representations of signals are widely used in ecoacoustics analyses aided by machine learning techniques. Most reviewed publications used frequency and time-frequency-based features extracted from several implementations of the Fourier Transform as the Discrete Fourier Transform (DFT) [54] , the Fast Fourier Transform (FFT) [44] , [35] , [23] , and the Short-Time Fourier Transform (STFT) [70] , [106] , [34] . The parameterization of the Fourier transform was described in almost all publications where it was used.…”

“…We also identified recent and novel characterization techniques. For example, [70] employed directional embedding for bird vocalizations; [109] used a fusion of frequency and time domain features for classifying anurans; [27] extracted features implementing a gray level co-occurrence matrix; and [92] combined features with the Recurrence Quantification Analysis (RQA) for urban sound identification. However, some authors have suggested that unsupervised learning frameworks are more suitable for soundscape analysis.…”

“…Nevertheless, deep learning represents a challenge due to the high volume of data needed to train models [93] . Moreover, supervised and semi-supervised models require labels for each sample or group of samples [70] , [89] .…”

“…Some notable examples include the GTZAN Corpus [113] , Xeno Canto [85] , [113] , and SERF [107] . Additionally, data from worldwide challenges, such as the Challenge on Detection and Classification of Acoustic Scenes and Events (DCASE), have played a significant role in advancing acoustic research [103] , [70] , [48] . These challenges cover a wide range of topics, including acoustic classification, acoustic scene detection, audio captioning, sound synthesis, and bioacoustics.…”