Computer vision cracks the leaf code

Abstract

Understanding the extremely variable, complex shape and venation characters of angiosperm leaves is one of the most challenging problems in botany. Machine learning offers opportunities to analyze large numbers of specimens, to discover novel leaf features of angiosperm clades that may have phylogenetic significance, and to use those characters to classify unknowns. Previous computer vision approaches have primarily focused on leaf identification at the species level. It remains an open question whether learning and classification are possible among major evolutionary groups such as families and orders, which usually contain hundreds to thousands of species each and exhibit many times the foliar variation of individual species. Here, we tested whether a computer vision algorithm could use a database of 7,597 leaf images from 2,001 genera to learn features of botanical families and orders, then classify novel images. The images are of cleared leaves, specimens that are chemically bleached, then stained to reveal venation. Machine learning was used to learn a codebook of visual elements representing leaf shape and venation patterns. The resulting automated system learned to classify images into families and orders with a success rate many times greater than chance. Of direct botanical interest, the responses of diagnostic features can be visualized on leaf images as heat maps, which are likely to prompt recognition and evolutionary interpretation of a wealth of novel morphological characters. With assistance from computer vision, leaves are poised to make numerous new contributions to systematic and paleobotanical studies.

Computer vision applied to herbarium specimens of German trees: testing the future utility of the millions of herbarium specimen images for automated identification

“…In plants, computer vision has been applied to images of 1907 fresh leaves belonging to 32 different plant species [6], images of fresh leaves of a few tree species [7], and images of fresh leaves of three legume species [8]. One study has applied machine learning to images of dead leaves, using (digitized images of) 7597 leaf clearings from 2001 genera of flowering plants to categorize leaf vein patterns [9]. Leaf clearings are leaves that have been chemically treated and preserved to show the veins.…”

Deep Learning for Multi-task Plant Phenotyping

“…Some cereal-specific examples include using expectation maximization to identify wheat streak mosaic virus [6]; Simplex Volume Maximisation to discover characteristic spectra in hyperspectral data of barley diseases [25]; and a support vector machine method to detect flowering rice in RGB images [11]. A support vector machine approach has also been used [27] to learn from SIFT features and a codebook generated over 7,500 images to classify higher plant taxa from images of leaves, achieving an accuracy of 72%.…”

A Remote Controlled Robotic Arm That Reads Barcodes and Handles Products

“…However, the small computers now available are powerful, use very little power and the software needed is simple and versatile. The use of these little devices is economical and they can easily be used for such applications as goods inventory [15,16].…”