What plant is that? Tests of automated image recognition apps for plant identification on plants from the British flora

Abstract

There has been a recent explosion in development of image recognition technology and its application to automated plant identification so it is timely to consider its potential for field botany. Nine free apps or websites for automated plant identification and suitable for use on mobile phones or tablet computers in the field were tested on a disparate set of 38 images of plants or parts of plants chosen from the higher plant flora of Britain and Ireland. There were large differences in performance with the best apps identifying more than 50% of samples tested to genus or better. Although the accuracy is good for some of the top-rated apps, for any quantitative biodiversity study or for ecological surveys there remains a need for validation by experts or against conventional floras. Nevertheless, the better-performing apps should be of great value to beginners and amateurs and may usefully stimulate interest in plant identification and nature. Potential uses of automated image recognition plant identification apps are discussed and recommendations made for their future use.

The Flora Incognita app – Interactive plant species identification

“…Results of the four referenced comparative studies show that Flora Incognita achieves state of the art identification accuracy. We attribute Flora Incognita's varying performance mainly to the studies' experimental protocol, for example, solely using single image observations (Jones, 2020; Shapovalov et al., 2020), unavailable or wrong geolocation preventing habitat analysis (Jones, 2020) and identifying non‐supported taxa (Jones, 2020; Schmidt & Steinecke, 2019). However, Jones (2020) still concludes that the Flora Incognita app is a very valuable tool even for botanists and ecologists during field studies.…”

“…Results of the four referenced comparative studies show that Flora Incognita achieves state of the art identification accuracy. We attribute Flora Incognita's varying performance mainly to the studies' experimental protocol, e.g., solely using single image observations (Jones, 2020;Shapovalov et al, 2020), unavailable or wrong geolocation preventing habitat analysis (Jones, 2020), and identifying non-supported taxa (Schmidt and Steinecke, 2019;Jones, 2020 (Jones, 2020)) to gain a repeatable experimental setup but thereby neglecting our multi-image and context analyses that have been demonstrated to substantially improve identification accuracy (Rzanny et al, 2019;Seeland and Mäder, 2021).…”

“…For these reasons, there has long been interest in developing automated species identification systems (Gaston & O'Neill, 2004; Wäldchen & Mäder, 2018b). Initial image‐based approaches were proposed 15 years ago but only now have become a reliable alternative to manual identifications (Affouard et al., 2017; Jones, 2020; Seeland et al., 2019; Wäldchen & Mäder, 2018a; Wäldchen et al., 2018). Recent boosts in data availability accompanied by substantial progress in machine learning algorithms, notably deep convolutional neural networks (CNNs; LeCun et al., 2015), pushed these approaches to a ‘production‐ready’ state.…”

Crowd‐sourced plant occurrence data provide a reliable description of macroecological gradients

“…Automated species identification is one promising avenue that has been discussed since the potential of machine learning for ecological applications has become evident (Gaston and O'Neill 2004). With the advent of deep learning methods (Goodfellow et al 2016) automated species identification reaches levels of accuracy comparable to human experts (Bonnet et al 2018, Wäldchen and Mäder 2018a, Jones 2020, Villon et al 2020). Today, multiple smartphone apps leverage such algorithms and enable users to identify e.g.…”

“…Today, multiple smartphone apps leverage such algorithms and enable users to identify e.g. plants, insects or birds directly in the field (Kumar et al 2012, Affouard et al 2017, Van Horn et al 2018, Wäldchen and Mäder 2018a, Jones 2020). The voluntarily shared ancillary information on time and location could soon turn such mobile observations into an invaluable resource for different monitoring tasks (Bonnet et al 2020).…”