GinJinn2: Object detection and segmentation for ecology and evolution

Abstract

Leveraging image data for ecological and evolutionary/systematic research typically requires substantial effort for data collection and preparation. The ability to automate time-consuming steps of this process, possibly along with further downstream analyses, for example, using programming languages like Python or R, can not only increase productivity, but also allow otherwise infeasible large-scale analyses. Recent advances in machine learning (ML), both on the soft-and hardware side, make it even possible to automate tasks that are difficult to solve by means of classically designed algorithms. Computer vision, in particular, has largely profited from deep

Automated extraction of seed morphological traits from images

“…Additionally, Traitor allows users to easily inspect the quality of image segmentation outputs. Therefore, our approach provides a more convenient and automated alternative to extract interpretable traits from seeds with diverse morphological attributes than user‐friendly open‐source solutions, such as ImageJ plugins (Loddo et al, 2022), image processing pipelines in Python (Gehan et al, 2017) and deep‐learning based models (Ott & Lautenschlager, 2022).…”

“…The first step for image processing is the segmentation of objects from the background, which is critical for the accuracy of all measurements. Well‐established methods, such as thresholding (Loddo et al, 2022; Olivoto, 2022) and deep learning (Ott & Lautenschlager, 2022; Schwartz & Alfaro, 2021), have been adapted for biologists with little or no coding experience, but their practicality is restricted when working with numerous species and structures. Thresholding can be inefficient when the seeds being measured are of a variety of sizes, surface structures and colours, causing optimal parameters to change, and even fail when seeds are either too small, glossy or have protruding structures.…”

From leaves to labels: Building modular machine learning networks for rapid herbarium specimen analysis with LeafMachine2

“…These frameworks are extremely flexible, well‐supported, and surprisingly approachable. As a result, many recent projects have also coalesced around these two frameworks with great success, including efforts to segment leaves (Younis et al, 2020; Triki et al, 2020, 2021; Guo et al, 2021; Hussein et al, 2021b; Gu et al, 2022; Ott and Lautenschlager, 2022), segment plant tissue (Love et al, 2021; Goëau et al, 2022; Milleville et al, 2023), isolate plant organs (Davis et al, 2020; Pearson et al, 2020; Triki et al, 2020; Ott and Lautenschlager, 2022), extract specimen label data (Milleville et al, 2023), isolate diseased or damaged leaf tissue (Kaur et al, 2022; Mu et al, 2022; Kavitha Lakshmi and Savarimuthu, 2023), measure bird skeletons (Weeks et al, 2023), isolate preserved snakes (Curlis et al, 2022), segment fossils (Panigrahi et al, 2022), or remotely monitor phenology (Mann et al, 2022). However, rather than relying on a single machine learning architecture to extract trait and archival data from specimens, we developed a modular framework of seven different machine learning algorithms that work in tandem to comprehensively process each image (Table 2, Figure 1).…”

“…To overcome these limitations, many groups turned to machine learning algorithms, typically some kind of convolutional neural network (CNN), which can categorize individual pixels as members of discrete classes (Ott et al, 2020; Weaver et al, 2020; Younis et al, 2020; Triki et al, 2020, 2021; Goëau et al, 2020, 2022; Guo et al, 2021; Love et al, 2021; Hussein et al, 2021b; Gu et al, 2022; Ott and Lautenschlager, 2022; Milleville et al, 2023). For the task of isolating and measuring individual leaves, semantic segmentation algorithms still lack the power to resolve complex situations (e.g., overlapping leaves) because they produce one mask that contains all leaf pixels and require postprocessing to obtain usable results (Weaver et al, 2020; Hussein et al, 2021b, 2022).…”

Extracting Quantitative Information from Images Taken in the Wild: A Case Study of Two Vicariants of the Ophrys aveyronensis Species Complex

“…Until recently, the tools to acquire and analyse such information were limited, so authors tended to use classical image processing software to obtain a proxy of this complex information-for example, by extracting the average colour of a picture by computing its mean values (and associated standard deviations) along the different colour space axes [12]. Fortunately, an increasing number of tools are being made available and provide accessible, automated, and consistent methods for digital image analysis [13][14][15][16][17][18][19][20]. In particular, recent R packages allow us to obtain information and carry on all downstream statistical analyses within the same computing environment [21][22][23][24][25].…”