Image-based plant phenotyping with incremental learning and active contours

Abstract

Plant phenotyping investigates how a plant's genome, interacting with the environment, affects the observable traits of a plant (phenome). It is becoming increasingly important in our quest towards efficient and sustainable agriculture. While sequencing the genome is becoming increasingly efficient, acquiring phenotype information has remained largely of low throughput. Current solutions for automated image-based plant phenotyping, rely either on semi-automated or manual analysis of the imaging data, or on expensive and proprietary software which accompanies costly hardware infrastructure. While some attempts have been made to create software applications that enable the analysis of such images in an automated fashion, most solutions are tailored to particular acquisition scenarios and restrictions on experimental design. In this paper we propose and test, a method for the segmentation and the automated analysis of time-lapse plant images from phenotyping experiments in a general laboratory setting, that can adapt to scene variability. The method involves minimal user interaction, necessary to establish the statistical experiments that may follow. At every time instance (i.e., a digital photograph), it segments the plants in images that contain many specimens of the same species. For accurate plant segmentation we propose a vector valued level set formulation that incorporates features of color intensity, local texture, and prior knowledge. Prior knowledge is incorporated using a plant appearance model implemented with Gaussian mixture models, which utilizes incrementally information from previously segmented instances. The proposed approach is tested on Arabidopsis plant images acquired with a static camera capturing many subjects at the same time. Our validation with ground truth segmentations and comparisons with state-of-the-art methods in the literature shows that the proposed method is able to handle images with complicated and changing background in an automated fashion. An accuracy of 96.7% (dice similarity coefficient) was observed, which was higher than other methods used for comparison. While here it was tested on a single plant species, the fact that we do not employ shape driven models and we do not rely on fully supervised classification (trained on a large dataset) increases the ease of deployment of the proposed solution for the study of different plant species in a variety of laboratory settings. Our solution will be accompanied by an easy to use graphical user interface and, to facilitate adoption, we will make the software available to the scientific community.

Leaf segmentation in plant phenotyping: a collation study

“…As we suspected whenever complications in the background are present, they do lower plant segmentation accuracy (explaining also large variation in performance). Possibly higher performance (and lower variability) can be obtained with methods relying on learned classifiers and active contour models [36]. Lower plant segmentation accuracy negatively affects leaf segmentation accuracy in almost all cases.…”

“…This is due to the variability in shape, pose, and appearance of leaves, but also due to lack of clearly discernible boundaries among overlapping leaves with typical imaging conditions where a top-view fixed camera is used. Several authors have dealt with the segmentation of a live plant from background to measure growth using unsupervised [17,25] and semi-supervised methods [36], but not of individual leaves. The use of color in combination with depth images or multiple images for supervised or unsupervised plant segmentation is also common practice [4,10,29,46,49,51,52,57].…”

“…In the first place, we obtained a binary segmentation of the plant objects in the scene in a computer-aided fashion. For Arabidopsis, we used the approach based on active contours described in [36], while for tobacco, a simple color-based approach for plant segmentation was used. The result of this segmentation was manually refined using raster graphics editing software.…”

Learning to Count Leaves in Rosette Plants

“…We compare with a counting via segmentation method [30] and recent density based methods [4,23]. Image data: We use three datasets, namely A1, A2, and A3, consisting of images showing top views on individual plants provided by the LCC CVPPP 2015 challenge organizers [25,32]. Images in A1 and A2 (approximately 500 × 500 pixels) are from Arabidopsis thaliana plant subjects, but in A1 are only from wild types (Col-0), while in A2 are also from four different mutant lines (plant identity is unknown in the images).…”

An interactive tool for semi-automated leaf annotation

“…This integration permits users to have a centralized repository for the data. The tool can be used to either create data for training larger machine-learning based applications or as part of a phenotyping analysis by integrating also phenotype extraction software [18,24].…”