Assessment of Error Rates in Acoustic Monitoring with the R package monitoR

Katz, Jonathan E.; Hafner, Sasha D.; Donovan, Therese M.

doi:10.1080/09524622.2015.1133320

Cited by 28 publications

(46 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some researchers have found that developing the training data set with clean references (less corrupted with noise) is helpful to avoid false positives (identification of any sound except the target bird sounds; Wildlife Acoustics (2011) and Boucher (2014)). Usually, song examples from the same environment where the test recordings will be collected are preferred (Katz et al 2016), as it is hoped that these will have similar noise profiles and less variations in calls. While recording clean close-range calls is possible with handheld (manual) recorders (because the recordist can get close to the individual birds and avoid noise by careful screening (Ruse et al 2016)), it is often more successful if the training data is based on the same type of recorder as will be used in practice.…”

Section: Noise Reductionmentioning

confidence: 99%

Automated birdsong recognition in complex acoustic environments: a review

Priyadarshani

Marsland

Castro

2018

Journal of Avian Biology

209

216

View full text Add to dashboard Cite

Conservationists are increasingly using autonomous acoustic recorders to determine the presence/absence and the abundance of bird species. Unlike humans, these recorders can be left in the field for extensive periods of time in any habitat. Although data acquisition is automated, manual processing of recordings is labour intensive, tedious, and prone to bias due to observer variations. Hence automated birdsong recognition is an efficient alternative. However, only few ecologists and conservationists utilise the existing birdsong recognisers to process unattended field recordings because the software calibration time is exceptionally high and requires considerable knowledge in signal processing and underlying systems, making the tools less user‐friendly. Even allowing for these difficulties, getting accurate results is exceedingly hard. In this review we examine the state‐of‐the‐art, summarising and discussing the methods currently available for each of the essential parts of a birdsong recogniser, and also available software. The key reasons behind poor automated recognition are that field recordings are very noisy, calls from birds that are a long way from the recorder can be faint or corrupted, and there are overlapping calls from many different birds. In addition, there can be large numbers of different species calling in one recording, and therefore the method has to scale to large numbers of species, or at least avoid misclassifying another species as one of particular interest. We found that these areas of importance, particularly the question of noise reduction, are amongst the least researched. In cases where accurate recognition of individual species is essential, such as in conservation work, we suggest that specialised (species‐specific) methods of passive acoustic monitoring are required. We also believe that it is important that comparable measures, and datasets, are used to enable methods to be compared.

show abstract

Section: Noise Reductionmentioning

confidence: 99%

Automated birdsong recognition in complex acoustic environments: a review

Priyadarshani

Marsland

Castro

2018

Journal of Avian Biology

209

216

View full text Add to dashboard Cite

show abstract

“…One of the challenges of using ARUs for ecological research and monitoring is the time required to extract target species detections from recordings (Shonfield and Bayne 2017). In response, automated signal recognition programs have been developed (e.g., de Oliveira et al 2015, Katz et al 2016. Automated acoustic species recognition is the process of training a computer to detect, recognize, and evaluate the acoustic signature of a target species' vocalization.…”

Section: Introductionmentioning

confidence: 99%

Recommendations for acoustic recognizer performance assessment with application to five common automated signal recognition programs

Knight¹,

Hannah²,

Foley³

et al. 2017

ACE

156

263

View full text Add to dashboard Cite

Automated signal recognition software is increasingly used to extract species detection data from acoustic recordings collected using autonomous recording units (ARUs), but there is little practical guidance available for ecologists on the application of this technology. Performance evaluation is an important part of employing automated acoustic recognition technology because the resulting data quality can vary with a variety of factors. We reviewed the bioacoustic literature to summarize performance evaluation and found little consistency in evaluation, metrics employed, or terminology used. We also found that few studies examined how score threshold, i.e., cutoff for the level of confidence in target species classification, affected performance, but those that did showed a strong impact of score threshold on performance. We used the lessons learned from our literature review and best practices from the field of machine learning to evaluate the performance of five readily-available automated signal recognition programs. We used the Common Nighthawk (Chordeiles minor) as our model species because it has simple, consistent, and frequent vocalizations. We found that automated signal recognition was effective for determining Common Nighthawk presence-absence and call rate, particularly at low score thresholds, but that occupancy estimates from the data processed with recognizers were consistently lower than from data generated by human listening and became unstable at high score thresholds. Of the five programs evaluated, our convolutional neural network (CNN) recognizer performed best, with recognizers built in Song Scope and MonitoR also performing well. The RavenPro and Kaleidoscope recognizers were moderately effective, but produced more false positives than the other recognizers. Finally, we synthesized six general recommendations for ecologists who employ automated signal recognition software, including what to use as a test benchmark, how to incorporate score threshold, what metrics to use, and how to evaluate efficiency. Future studies should consider our recommendations to build a body of literature on the effectiveness of this technology for avian research and monitoring. Recommandations pour l'évaluation des performances de reconnaissance acoustique et application à cinq programmes courants de reconnaissance automatisée de signaux sonores RÉSUMÉ. Les logiciels de reconnaissance automatisée de signaux sonores sont de plus en plus utilisés pour extraire les données de détection des espèces d'enregistrements acoustiques récoltés au moyen d'unités d'enregistrement autonomes (ARU en anglais), mais il existe peu d'instructions pratiques sur l'utilisation de cette technologie pour les écologistes. L'évaluation de la performance est une étape importante dans l'utilisation d'une technologie de reconnaissance acoustique automatisée parce que la qualité des résultats peut varier en fonction de divers facteurs. Nous avons passé en revue la littérature sur la bioacoustique afin de résumer les critères d'évaluation de ...

show abstract

“…Knight et al (2017) provide an overview of underlying principles and performance benchmarking of five readily available species recognition programs. Among these programs, the template-based MonitoR software (Katz et al 2016) is particularly promising, because it is a package implemented in R (https:// www.r-project.org/), a free software project becoming increasingly popular among biologists. In addition, R contains the seewave package (Sueur et al 2008a), designed for sound analysis and synthesis.…”

Section: The Way Forward: Algorithms For Acoustic Profilingmentioning

confidence: 99%

Acoustic profiling of Orthoptera: present state and future needs

Riede¹

2018

JOR

View full text Add to dashboard Cite

Bioacoustic monitoring and classification of animal communication signals has developed into a powerful tool for measuring and monitoring species diversity within complex communities and habitats. The high number of stridulating species among Orthoptera allows their detection and classification in a non-invasive and economic way, particularly in habitats where visual observations are difficult or even impossible, such as tropical rainforests. Major sound archives were queried for Orthoptera songs, with special emphasis on usability as reference training libraries for computer algorithms. Orthoptera songs are highly stereotyped, reliable taxonomic features. However, exploitation of songs for acoustic profiling is limited by the small number of reference recordings: existing song libraries represent only about 1000 species, mainly from Europe and North America, covering less than 10% of extant stridulating Orthoptera species. Available databases are fragmented and lack tools for song annotation and efficient feature-based searching. Results from recent bioacoustic surveys illustrate the potential of the method, but also the challenges and bottlenecks impeding further progress. A major problem is time-consuming data analysis of recordings. Computer-aided identification software exists for classification and identification of cricket and grasshopper songs, but these tools are still far from practical for field application.A framework for acoustic profiling of Orthoptera should consist of the following components: (1) Protocols for standardized acoustic sampling, at species and community levels, using acoustic data loggers for autonomous long-term recordings; (2) Open access to and efficient management of song data and voucher specimens, involving the Orthoptera Species File (OSF) and Global Biodiversity Information Facility (GBIF); (3) An infrastructure for automatized analysis and song classification; and (4) Complementation and improvement of Orthoptera sound libraries using OSF as the taxonomic backbone and repository for representative song recordings. Taxonomists should be encouraged, or even obliged, to deposit original recordings, particularly if they form part of species descriptions or revisions. Citation: Riede K (2018) Acoustic profiling of Orthoptera: present state and future needs. Journal of Orthoptera Research 27(2): 203-215. https://doi.

show abstract

Assessment of Error Rates in Acoustic Monitoring with the R package monitoR

Cited by 28 publications

References 34 publications

Automated birdsong recognition in complex acoustic environments: a review

Automated birdsong recognition in complex acoustic environments: a review

Recommendations for acoustic recognizer performance assessment with application to five common automated signal recognition programs

Acoustic profiling of Orthoptera: present state and future needs

Contact Info

Product

Resources

About