FlowerPhenoNet: Automated Flower Detection from Multi-View Image Sequences Using Deep Neural Networks for Temporal Plant Phenotyping Analysis

Abstract

A phenotype is the composite of an observable expression of a genome for traits in a given environment. The trajectories of phenotypes computed from an image sequence and timing of important events in a plant’s life cycle can be viewed as temporal phenotypes and indicative of the plant’s growth pattern and vigor. In this paper, we introduce a novel method called FlowerPhenoNet, which uses deep neural networks for detecting flowers from multiview image sequences for high-throughput temporal plant phenotyping analysis. Following flower detection, a set of novel flower-based phenotypes are computed, e.g., the day of emergence of the first flower in a plant’s life cycle, the total number of flowers present in the plant at a given time, the highest number of flowers bloomed in the plant, growth trajectory of a flower, and the blooming trajectory of a plant. To develop a new algorithm and facilitate performance evaluation based on experimental analysis, a benchmark dataset is indispensable. Thus, we introduce a benchmark dataset called FlowerPheno, which comprises image sequences of three flowering plant species, e.g., sunflower, coleus, and canna, captured by a visible light camera in a high-throughput plant phenotyping platform from multiple view angles. The experimental analyses on the FlowerPheno dataset demonstrate the efficacy of the FlowerPhenoNet.

Drought stress prediction and propagation using time series modeling on multimodal plant image sequences

“…It has three watering stations with a balance that can add water to a target weight or specific volume and records the specific quantity of water added daily. The images of the greenhouse with plants placed on the automated conveyor belt, the watering station, and plants entering into the imaging cabinets are available in ( Das Choudhury et al., 2018 ; Das Choudhury et al., 2022 ). The cameras installed in the four imaging chambers are (a) visible light - side view and top view (Prosilica GT6600 29 megapixel camera with a Gigabit Ethernet interface 1 ), (b) infrared - side view and top view (Pearleye p-030 LWIR), (c) fluorescent - side view and top view (Basler Scout scA1400-17gm/gc), and (d) hyperspectral - side view (Headwall Hyperspec Inspector x-vnir 2 ) and near-infrared - top view (Goldeye p-008 SWIR), respectively.…”

“…Deep neural networks have been successfully employed in high throughput temporal plant phenotyping for a variety of applications ( Bashyam et al., 2021 ; Zheng et al., 2021 ; Das Choudhury et al., 2022 ). The method in ( Das Choudhury et al., 2022 ) performs automated flower detection from multi-view image sequences to determine a set of novel phenotypes, e.g., the emergence time of the first flower, the total number of flowers present in the plant at a given time, flower growth trajectory, and blooming trajectory. A graph theoretic approach has been used by ( Bashyam et al., 2021 ) to detect and track individual leaves of a maize plant for automated growth stage monitoring.…”

Pollinator‐assisted plant phenotyping, selection, and breeding for crop resilience to abiotic stresses

“…Although increasing interest exists in predicting yield through flowering time and bloom (Das Choudhury et al., 2022; Han et al., 2021; Zhang et al., 2020), floral traits are rarely analyzed in these digital‐based platforms because of difficulties in procuring high spatial–temporal resolution for flowers (McCabe & Tester, 2021; Mochida et al., 2020). Drone‐borne imaging systems assisted by machine learning are being developed to increase accurate flower detection in field crops aiming at predicting yield through flowering time and/or intensity, but it requires continuous empirical algorithmic development for each species or variety studied (e.g., specific flower color and shape, variable baseline noise) and has, to date, mainly been applied to populations of relatively low genetic variability (Pieruschka & Schurr, 2019).…”