Computer vision approach to characterize size and shape phenotypes of horticultural crops using high-throughput imagery

Abstract

For many horticultural crops, variation in quality (e.g., shape and size) contribute significantly to the crop's market value. Metrics characterizing less subjective harvest quantities (e.g., yield and total biomass) are routinely monitored. In contrast, metrics quantifying more subjective crop quality characteristics such as ideal size and shape remain difficult to characterize objectively at the production-scale due to the lack of modular technologies for high-throughput sensing and computation. Several horticultural crops are sent to packing facilities after having been harvested, where they are sorted into boxes and containers using high-throughput scanners. These scanners capture images of each fruit or vegetable being sorted and packed, but the images are typically used solely for sorting purposes and promptly discarded. With further analysis, these images could offer unparalleled insight on how crop quality metrics vary at the industrial production-scale and provide further insight into how these characteristics translate to overall market value. At present, methods for extracting and quantifying quality characteristics of crops using images generated by existing industrial infrastructure have not been developed. Furthermore, prior studies that investigated horticultural crop quality metrics, specifically of size and shape, used a limited number of samples, did not incorporate deformed or non-marketable samples, and did not use images captured from high-throughput systems. In this work, using sweetpotato (SP) as a use case, we introduce a computer vision algorithm for quantifying shape and size characteristics in a high-throughput manner. This approach generates 3D model of SPs from two 2D images captured by an industrial sorter 90 degrees apart and extracts 3D shape features in a few hundred milliseconds. We applied the 3D reconstruction and feature extraction method to thousands of image samples to demonstrate how variations in shape features across sweetptoato cultivars can be quantified. We created a sweetpotato shape dataset containing sweetpotato images, extracted shape features, and qualitative shape types (U.S. No. 1 or Cull). We used this dataset to develop a neural network-based shape classifier that was able to predict Cull vs. U.S. No. 1 sweetpotato with 84.59% accuracy. In addition, using univariate Chi-squared tests and random forest, we identified the most important features for determining qualitative shape (U.S. No. 1 or Cull) of the sweetpotatoes. Our study serves as the first step towards enabling big data analytics for sweetpotato agriculture. The methodological framework is readily

Image-Based High-Throughput Phenotyping in Horticultural Crops

“…Although visual evaluation is a widely used nondestructive method for grading and sorting in horticultural crops, utilization of high-throughput phenotyping platforms is essential to obtain robust, faster, and objective results [ 14 ]. Nowadays, quantitative measurement of these traits (color, size, shape, and surface texture) using image analysis is increasingly used in different horticultural crops [ 13 , 84 , 85 , 86 , 87 ].…”

“…The application of computer vision for shape quantification using images of sweet potatoes has shown that shape features, length-to-width ratio, curvature, cross-section roundness, and cross-sectional diameters, are highly predictive of shape classes. A neural network-based shape classifier was able to predict marketable (high market value) and non-marketable sweet potato classes with 84.59% accuracy [ 13 ]. In most food industries, quality is mainly assessed by experts based on subjective evaluation, which is very slow and inconsistent.…”

“…The market value of horticultural products is highly dependent on external appearance (color, shape, size, texture) and internal (soluble solid content and firmness) quality attributes. The status of these quality attributes changes over time during maturity, ripening, and postharvest storage and should be routinely monitored [ 13 , 14 ]. Currently, external quality attributes are mostly evaluated using visual inspection in the horticulture chain, which is slow and subjective.…”

Emerging trends in the agri-food sector: Digitalisation and shift to plant-based diets

“…Big Data Analytics (BDA) is the process of using sophisticated analytics on BD ( Ciccullo et al, 2022 ). The most current or potential applications of BD in vegetables, fruits and other plant-based sectors include optimal planting of fruit trees using data extracted from satellite and unmanned aerial vehicle imagery ( Saldana Ochoa and Guo, 2019 ), characterisation of size and shape phenotypes of horticultural crops using high throughput imagery ( Haque et al, 2021 ), improvement of controlled environment agriculture, such as soilless hydroponics and others for vegetable and fruit farming ( Ragaveena et al, 2021 ), and mitigating post-harvest losses and managing fruit and vegetable quality through machine learning ( Singh et al, 2022 ).…”

Convolutional neural networks in the qualitative improvement of sweet potato roots

“…The main challenge for commercializing this vegetable is associated with its market value since it is strongly dependent on the qualitative characteristics of the roots, such as the shape 5 . Hence, consumers prefer products that have adequate commercial standards and good appearance.…”