Extensive data engineering to the rescue: building a multi-species katydid detector from unbalanced, atypical training datasets

Abstract

Passive acoustic monitoring (PAM) is a powerful tool for studying ecosystems. However, its effective application in tropical environments, particularly for insects, poses distinct challenges. Neotropical katydids produce complex species-specific calls, spanning mere milliseconds to seconds and spread across broad audible and ultrasonic frequencies. However, subtle differences in inter-pulse intervals or central frequencies are often the only discriminatory traits. These extremities, coupled with low source levels and susceptibility to masking by ambient noise, challenge species identification in PAM recordings. This study aimed to develop a deep learning-based solution to automate the recognition of 31 katydid species of interest in a biodiverse Panamanian forest with over 80 katydid species. Besides the innate challenges, our efforts were also encumbered by a limited and imbalanced initial training dataset comprising domain-mismatched recordings. To overcome these, we applied rigorous data engineering, improving input variance through controlled playback re-recordings and by employing physics-based data augmentation techniques, and tuning signal-processing, model and training parameters to produce a custom well-fit solution. Methods developed here are incorporated into Koogu, an open-source Python-based toolbox for developing deep learning-based bioacoustic analysis solutions. The parametric implementations offer a valuable resource, enhancing the capabilities of PAM for studying insects in tropical ecosystems. This article is part of the theme issue ‘Towards a toolkit for global insect biodiversity monitoring’.

Multi-year soundscape recordings and automated call detection reveals varied impact of moonlight on calling activity of neotropical forest katydids

“…The full training set consisted of nearly 4500 calls, ranging from 20–680 exemplars per species. For full details of model development, see Madhusudhana et al [ 41 ].…”

Towards a toolkit for global insect biodiversity monitoring

“…Madhusudhana et al . [ 54 ] present one solution for this problem by developing a method for training a deep learning model using sparse data of poorly known species. Another option for expanding sets of available training data will be the sharing of annotated species- and group-specific insect recordings by scientists and citizens online.…”

“…This variability in properties will also need to be accounted for by collecting reference data under various environmental conditions, and/or using sophisticated modelling approaches. Given that most insect species are rarely observed, new methods are required to include these species despite the scarcity of records (see [ 54 ]). To accommodate the massive growth of these libraries, improved matching algorithms are needed [ 29 ].…”

Acoustic monitoring for tropical insect conservation

“…Sugai et al 2019, Gibb et al 2019) in combination with curated multimedia repositories with references to voucher specimens, as well as pictures and videos provided by citizen scientists. In addition, considerable progress has been made in automatic song recognition, which allows identification of prominent songsters in large datasets from an increasing number of acoustic monitoring sites (Faiß and Stowell 2023; Madhusudhana et al 2024).…”

“…1). In a recent study, Madhusudhana et al (2024) used a variety of data augmentation techniques together with machine learning to explore automated recognition of the calls of 31 katydid species from field soundscape recordings on Barro Colorado Island, Panama. Given the diversity of insect species and soundscapes worldwide, such studies need to be replicated and the performance of automated algorithms evaluated and improved across the tropics.…”