Deep machine learning provides state-of-the-art performance in image-based plant phenotyping

Abstract

In plant phenotyping, it has become important to be able to measure many features on large image sets in order to aid genetic discovery. The size of the datasets, now often captured robotically, often precludes manual inspection, hence the motivation for finding a fully automated approach. Deep learning is an emerging field that promises unparalleled results on many data analysis problems. Building on artificial neural networks, deep approaches have many more hidden layers in the network, and hence have greater discriminative and predictive power. We demonstrate the use of such approaches as part of a plant phenotyping pipeline. We show the success offered by such techniques when applied to the challenging problem of image-based plant phenotyping and demonstrate state-of-the-art results (>97% accuracy) for root and shoot feature identification and localization. We use fully automated trait identification using deep learning to identify quantitative trait loci in root architecture datasets. The majority (12 out of 14) of manually identified quantitative trait loci were also discovered using our automated approach based on deep learning detection to locate plant features. We have shown deep learning–based phenotyping to have very good detection and localization accuracy in validation and testing image sets. We have shown that such features can be used to derive meaningful biological traits, which in turn can be used in quantitative trait loci discovery pipelines. This process can be completely automated. We predict a paradigm shift in image-based phenotyping bought about by such deep learning approaches, given sufficient training sets.

Segmentation of roots in soil with U-Net

“…Effective feature engineering for plant phenotyping requires a practitioner with a broad skill-set as they must have sufficient knowledge of both image analysis, machine learning and plant physiology [40]. Not only is it difficult to find the optimal description of the data but the features found may limit the performance of the system to specific datasets [41]. With feature engineering approaches, domain knowledge is expressed in the feature extraction code so further programming is required to re-purpose the system to new datasets.…”

“…CNNs have now established their dominance on almost all recognition and detection tasks [42][43][44][45]. They have also been used to segment roots from soil in X-ray tomography [46] and to identify the tips of wheat roots grown in germination paper growth pouches [41]. CNNs have an ability to transfer well from one task to another, requiring less training data for new tasks [47].…”

A deep learning-based approach for high-throughput hypocotyl phenotyping

“…In recent years, the introduction of deep learning and convolutional neural networks revolutionized computer vision-based research, making the automation of various tasks and precise high-throughput phenotyping available for many disciplines. In plant biology, several advances have been made with these methods regarding qualitative phenotyping (Pound et al, 2017;Namin et al, 2018;Pineda et al, 2018;Singh et al, 2018;Ramcharan et al, 2019). With these tools however, quantitative phenotypic traits can also be assessed as we demonstrated in this work.…”

“…During this part, information is compressed and can be lost during the pooling steps. For tasks such as image classification, this is not a problem (Pound et al, 2017). However, for semantic segmentation tasks, the network is required to reconstruct the pixel-level segmentation mask, which is achieved by upsampling the feature-level representation.…”

“…This makes its performance exceptionally good, and given enough data, a well-trained neural network can generalize for a wide range of datasets. For plant phenotyping, these developments yielded advances for instance in trait identification and genotype/phenotype classification (Pound et al, 2017;Namin et al, 2018).…”