Estimation of Greenhouse Lettuce Growth Indices Based on a Two-Stage CNN Using RGB-D Images

Abstract

Growth indices can quantify crop productivity and establish optimal environmental, nutritional, and irrigation control strategies. A convolutional neural network (CNN)-based model is presented for estimating various growth indices (i.e., fresh weight, dry weight, height, leaf area, and diameter) of four varieties of greenhouse lettuce using red, green, blue, and depth (RGB-D) data obtained using a stereo camera. Data from an online autonomous greenhouse challenge (Wageningen University, June 2021) were employed in this study. The data were collected using an Intel RealSense D415 camera. The developed model has a two-stage CNN architecture based on ResNet50V2 layers. The developed model provided coefficients of determination from 0.88 to 0.95, with normalized root mean square errors of 6.09%, 6.30%, 7.65%, 7.92%, and 5.62% for fresh weight, dry weight, height, diameter, and leaf area, respectively, on unknown lettuce images. Using red, green, blue (RGB) and depth data employed in the CNN improved the determination accuracy for all five lettuce growth indices due to the ability of the stereo camera to extract height information on lettuce. The average time for processing each lettuce image using the developed CNN model run on a Jetson SUB mini-PC with a Jetson Xavier NX was 0.83 s, indicating the potential for the model in fast real-time sensing of lettuce growth indices.

Lettuce Production in Intelligent Greenhouses—3D Imaging and Computer Vision for Plant Spacing Decisions

“…The product of the multiplied area per lettuce head with the maximum height resulted in the highest correlation coefficient with fresh weight ( Appendix A Table A3 ). Three papers using the [ 50 , 51 , 52 ] dataset had a RMSE up to 25.3. As indicated, we obtained a lower accuracy, however, we should take into account that the datasets are not fully comparable.…”

“…The ability of networks to learn plant features from single lettuce images can be determined by the recently published lettuce dataset [ 49 ]. At the moment, three papers have been published, obtaining high accuracy to estimate fresh weight from the images with a Root Mean Squared Error (RMSE) up to 25.3 g [ 50 , 51 , 52 ].…”

Classification and Forecasting of Water Stress in Tomato Plants Using Bioristor Data

“…[6] Tomato images leaf miner DL class. [7] Lettuce image growth indices DL pred. [8] Apple images fruit maturation ML class.…”

“…The usage of artificial intelligence, and specifically machine learning techniques, in the agricultural field has been an increasing topic in the last ten years [22]. For example, in [6] the authors try to detect leaf miners in tomato plants by applying two types of deep neural network to classify and segment plant images, while in [7] a convolutional neural network is used to predict some growth indices of lettuce plants, always using image data. In [8] the target crop regards apples and in particular the classification of their maturation degree in order to facilitate the work of picking robots.…”

Temporal forecasting of plant height and canopy diameter from RGB images using a CNN-based regression model for ornamental pepper plants (Capsicum spp.) growing under high-temperature stress

“…In this sense, arti cial vision techniques can be useful tools within genetic breeding programs as they can enable the prediction of morphological parameters of plants in advance before the evaluation date. For example, arti cial intelligence techniques, including those associated with convolutional neural networks (CNNs), have been adopted for this kind of prediction in other species, such as Chinese mahogany (Liu et al, 2021), passion fruit (Tu et al, 2020), lettuce (Gang et al, 2022), among others. In the literature, some studies can be found using arti cial neural networks for processing images in order to estimate the number of outdoor ornamental plants (Bayraktar et al, 2020).…”