Going further with model verification and deep learning

Abstract

In our recent review paper aiming to introduce deep learning to ecologists, we presented a workflow describing the steps required to create a deep learning model. This figure did not present some of the following steps of model use such as model verification. By ensuring model adequacy, model verification is an important step after model creation in order to answer ecological questions. Adding model verification to a deep learning model development workflow can raise some new issues such as detecting the difference among the multiple datasets or what to do when model verification fails. In the spirit of our previous review, we identify some questions users trying to verify their deep learning model can have and try to find, for each, a solution to help them navigate the steps of deep learning model testing. We provide an additional cheat sheet to quickly help answer common questions regarding using model verification and deep learning. We hope these resources help stimulate further synthesis and coherence in the use of deep learning models in ecology.

A deep Generative Artificial Intelligence system to predict species coexistence patterns

“…= 6,840 possible combinations that should be replicated. Third, proper training and validation are needed to decide whether the model is good enough to be used (see Christin et al, 2021). To further facilitate the application of this method to other datasets, the code used in this study has been made accessible in a repository (https://github.com/jegar cian/AI4Ec ology) and detailed steps added to the documentation.…”

Use of object detection in camera trap image identification: assessing a method to rapidly and accurately classify human and animal detections for research and application in recreation ecology

“…active and transfer learning, Beery et al, 2020; Norouzzadeh et al, 2021), a pragmatic compromise between entirely manual and entirely automated classification of camera trap data is the use of object detection models to assist in filtering images into relevant classes, allowing increased efficiency for manual processing (Beery et al, 2018; Beery, Morris, & Yang, 2019; Greenberg et al, 2019). A key limit to the widespread adoption of such tools by ecologists is a lack of external validation of their performance, slowing adoption of new technologies into practice (Christin et al, 2021).…”

“…While MegaDetector is presented as an effective tool for accelerating data processing, quantification of performance is crucial prior to widespread implementation into real-world workflows (Christin et al, 2021). A key consideration in the evaluation of machine learning model outputs is that these models do not truly “categorize” objects in images, but rather provide a confidence value pertaining to the likelihood of that image containing each object class.…”

“…A key limit to the widespread adoption of such tools by ecologists is a lack of external validation of their performance, slowing adoption of new technologies into practice (Christin et al, 2021).…”

“…Automated processing of images into these classes has the potential to be conducted far faster than could be completed manually by humans and is generally limited only by computer processing speeds (Microsoft, 2020). While MegaDetector is presented as an effective tool for accelerating data processing, quantification of performance is crucial prior to widespread implementation into real-world workflows (Christin et al, 2021). Here we explore the use of MegaDetector in streamlining camera trap data processing, with a specific focus on projects wishing to quantify large numbers of human images in the context of recreation ecology.…”