Call recognition and individual identification of fish vocalizations based on automatic speech recognition: An example with the Lusitanian toadfish

Abstract

The study of acoustic communication in animals often requires not only the recognition of species specific acoustic signals but also the identification of individual subjects, all in a complex acoustic background. Moreover, when very long recordings are to be analyzed, automatic recognition and identification processes are invaluable tools to extract the relevant biological information. A pattern recognition methodology based on hidden Markov models is presented inspired by successful results obtained in the most widely known and complex acoustical communication signal: human speech. This methodology was applied here for the first time to the detection and recognition of fish acoustic signals, specifically in a stream of round-the-clock recordings of Lusitanian toadfish (Halobatrachus didactylus) in their natural estuarine habitat. The results show that this methodology is able not only to detect the mating sounds (boatwhistles) but also to identify individual male toadfish, reaching an identification rate of ca. 95%. Moreover this method also proved to be a powerful tool to assess signal durations in large data sets. However, the system failed in recognizing other sound types.

MatlabHTK: a simple interface for bioacoustic analyses using hidden Markov models

“…), fish (Vieira et al . ), birds (Somervuo, Harma & Fagerlund ; Brandes ; Trifa et al . ; Potamitis et al .…”

“…Several sound processing algorithms that were primarily designed for speech recognition have successfully been applied by experts to analyse biotic vocalisations (Adi, Johnson & Osiejuk 2010). In particular, the Hidden Markov Toolkit (HTK, Young et al 2006), which implements hidden Markov model (HMM) approaches (Rabiner 1989), have widely been used in studies of insects (Aide et al 2013), fish (Vieira et al 2015), birds (Somervuo, Harma & Fagerlund 2006;Brandes 2008;Trifa et al 2008;Potamitis et al 2014) and mammals (Scheifele et al 2015). Because HMMs represent the signal statistically, they are flexible and can handle variations in a species vocalisations under various noise levels (Ren et al 2009).…”

Long-term acoustic monitoring of fish calling provides baseline estimates of reproductive timelines in the May River estuary, southeastern USA

“…However, there are not many studies where signal detectors have been created to detect fish acoustic signals, especially successful ones that can identify fish calling amidst a noisy background. Recently, automatic call recognition was applied to identify Lusitanian toadfish Halobatrachus didactylus vocalizations in Portugal and red grouper Epinephelus morio calls on the West Florida Shelf, but these soundscapes are quieter than estuaries and the diversity of soniferous fishes was less than in the May River (Wall et al 2014, Vieira at al. 2015.…”

Automatic fish sounds classification

“…Specifically, studies on environmental sounds (i.e., natural, animal, human sources), speech, and music have been conducted, and reviewed. Among the very few studies dealing with fish sounds classification, we can mention (Noda et al, 2016;Vieira et al, 2015) and (Sattar et al, 2016) in which results are promising even if the tools are not all tested in an applicative context of underwater monitoring. The classification of bioacoustics data is being developed in the literature, such as the sounds of sea mammals (Zaugg et al, 2010), bats, frogs (Chen et al, 2012;Huang et al, 2009), birds (Acevedo et al, 2009;Fagerlund, 2007;Tyagi et al, 2006) and other animals (Mitrovic et al, 2006).…”

“…The success of MFCCs in speechrelated studies is such that they are now considered as a reference set of features for acoustic classification purposes in general. The state of the art in automatic classification of fish sounds is very limited but both (Vieira et al, 2015) and (Noda et al, 2016) have used MFCCs for fish call recognition or fish individuals classification. In bioacoustics, MFCCs have been used by (Bedoya et al, 2014) for anuran sounds classification, by (Fagerlund, 2007) for bird call recognition, and by (Tyagi et al, 2006) for bird species recognition.…”