Enlisting 3D Crop Models and GANs for More Data Efficient and Generalizable Fruit Detection

Fei, Zhenghao; Olenskyj, Alexander G.; Bailey, Brian N.; Earles, Mason

doi:10.1109/iccvw54120.2021.00147

Cited by 16 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, we found that we can generate accurate models using 5 annotated slices from at least 3 unique leaf scans. This discrepancy is likely the result of the consistent image collection settings (resolution) on the same imaging platform (Beamline 8.3.2) which simplifies the learning process ( Fei et al, 2021 ; Silwal et al, 2021 ). Our results are comparable to those previously achieved by Théroux-Rancourt et al (2020) on X-ray CT images of plant leaves.…”

Section: Discussionmentioning

confidence: 99%

A workflow for segmenting soil and plant X-ray computed tomography images with deep learning in Google’s Colaboratory

et al. 2022

View full text Add to dashboard Cite

X-ray micro-computed tomography (X-ray μCT) has enabled the characterization of the properties and processes that take place in plants and soils at the micron scale. Despite the widespread use of this advanced technique, major limitations in both hardware and software limit the speed and accuracy of image processing and data analysis. Recent advances in machine learning, specifically the application of convolutional neural networks to image analysis, have enabled rapid and accurate segmentation of image data. Yet, challenges remain in applying convolutional neural networks to the analysis of environmentally and agriculturally relevant images. Specifically, there is a disconnect between the computer scientists and engineers, who build these AI/ML tools, and the potential end users in agricultural research, who may be unsure of how to apply these tools in their work. Additionally, the computing resources required for training and applying deep learning models are unique, more common to computer gaming systems or graphics design work, than to traditional computational systems. To navigate these challenges, we developed a modular workflow for applying convolutional neural networks to X-ray μCT images, using low-cost resources in Google’s Colaboratory web application. Here we present the results of the workflow, illustrating how parameters can be optimized to achieve best results using example scans from walnut leaves, almond flower buds, and a soil aggregate. We expect that this framework will accelerate the adoption and use of emerging deep learning techniques within the plant and soil sciences.

show abstract

Section: Discussionmentioning

confidence: 99%

A workflow for segmenting soil and plant X-ray computed tomography images with deep learning in Google’s Colaboratory

et al. 2022

View full text Add to dashboard Cite

show abstract

“…This has been noted in previous works [17,13], in which the laborious nature of image labeling was specifically noted. This labeling effort becomes increasingly burdensome as the variability of images conditions increases, pointing to a need for more data efficient methods for labeling, such as in Fei et al [46]. In this work, the requirement for labeling was sidestepped in the regression approach.…”

Section: Yield Estimation Using Object Detectionmentioning

confidence: 99%

End-to-end deep learning for directly estimating grape yield from ground-based imagery

Olenskyj,

Sams,

Fei

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Yield estimation prior to harvest is a powerful tool in vineyard management, as it allows growers to fine-tune management practices to optimize yield and quality. However, yield estimation is currently performed using manual sampling, which is time-consuming and imprecise. This study demonstrates the applicability of nondestructive proximal imaging combined with deep learning for yield estimation in vineyards. Continuous image data collection using a vehicle-mounted sensing kit combined with collection of ground truth yield data at harvest using a commercial yield monitor allowed for the generation of a large dataset of 23,581 yield points and 107,933 images. Moreover, this study was conducted in a commercial vineyard which was mechanically managed, representing a challenging environment for image analysis but a common set of conditions in the California Central Valley. Three model architectures were tested: object detection, CNN regression, and transformer models. The object detection model was trained on hand-labeled images to localize grape bunches, and detections were either counted or their pixel count was summed to obtain a metric which was correlated to grape yield. Conversely, regression models were trained end-to-end to directly predict grape yield from image data without the need for hand labeling. Results demonstrated that both a transformer model as well as the object detection model with pixel area processing performed comparably, with a mean absolute percent error of 18% and 18.5%, respectively on a representative holdout dataset. Saliency mapping was used to demonstrate the attention of the CNN regression model was localized near the predicted location of grape bunches, as well as on the top of the grapevine canopy. Overall, the study demonstrated the applicability of proximal imaging and deep learning for prediction of grapevine yield on a large scale. Additionally, the end-to-end modeling approach was able to perform comparably to the object detection approach while eliminating the need for hand-labeling.

show abstract

“…Thus, they presented a method for producing high-quality pseudo-labels by combining self-supervised and transfer learning. On the other hand, Fei et al [ 20 ], in order to enable more data-efficient and generalizable neural network models in agriculture, proposed a method that generates photorealistic agricultural images using a semantically constrained Generative Adversarial Network (GAN) to preserve fruit position and geometry. The method can significantly speed up the domain adaption process, increase performance, and decrease labeling requirements.…”

Section: Related Workmentioning

confidence: 99%

Mushroom Detection and Three Dimensional Pose Estimation from Multi-View Point Clouds

Retsinas

Efthymiou

Anagnostopoulou

et al. 2023

Sensors

View full text Add to dashboard Cite

Agricultural robotics is an up and coming field which deals with the development of robotic systems able to tackle a multitude of agricultural tasks efficiently. The case of interest, in this work, is mushroom collection in industrial mushroom farms. Developing such a robot, able to select and out-root a mushroom, requires delicate actions that can only be conducted if a well-performing perception module exists. Specifically, one should accurately detect the 3D pose of a mushroom in order to facilitate the smooth operation of the robotic system. In this work, we develop a vision module for 3D pose estimation of mushrooms from multi-view point clouds using multiple RealSense active–stereo cameras. The main challenge is the lack of annotation data, since 3D annotation is practically infeasible on a large scale. To address this, we developed a novel pipeline for mushroom instance segmentation and template matching, where a 3D model of a mushroom is the only data available. We evaluated, quantitatively, our approach over a synthetic dataset of mushroom scenes, and we, further, validated, qualitatively, the effectiveness of our method over a set of real data, collected by different vision settings.

show abstract

Enlisting 3D Crop Models and GANs for More Data Efficient and Generalizable Fruit Detection

Cited by 16 publications

References 17 publications

A workflow for segmenting soil and plant X-ray computed tomography images with deep learning in Google’s Colaboratory

A workflow for segmenting soil and plant X-ray computed tomography images with deep learning in Google’s Colaboratory

End-to-end deep learning for directly estimating grape yield from ground-based imagery

Mushroom Detection and Three Dimensional Pose Estimation from Multi-View Point Clouds

Contact Info

Product

Resources

About