Real-time detection of broccoli crops in 3D point clouds for autonomous robotic harvesting

Montes, Hector A.; Louedec, Justin Le; Cielniak, Grzegorz; Duckett, Tom

doi:10.1109/iros45743.2020.9341381

Cited by 11 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiview approaches can be combined with deep learning techniques to segment and build a 3D point cloud of the plant. Such 3D point clouds offer higher precision when segmenting plant parts into stem and leaves, including individual instances of leaves, when compared to 2D segmentation, which can occasionally classify similar-looking background pixels as foreground (plant). ,, Magistri et al extended the advantages of a 3D point cloud to a temporal association, by using semantic segmentation via an SVM to extract correspondences between point clouds, allowing for phenotype tracking over plant growth …”

Section: Resultsmentioning

confidence: 99%

“…Such 3D point clouds offer higher precision when segmenting plant parts into stem and leaves, including individual instances of leaves, when compared to 2D segmentation, which can occasionally classify similarlooking background pixels as foreground (plant). 103,157,158 Magistri et al extended the advantages of a 3D point cloud to a temporal association, by using semantic segmentation via an SVM to extract correspondences between point clouds, allowing for phenotype tracking over plant growth. 159 To summarize, more sophisticated models which tightly couple shoot dynamics with other components are needed for DCEA.…”

Section: 33mentioning

confidence: 99%

See 1 more Smart Citation

Dynamically Controlled Environment Agriculture: Integrating Machine Learning and Mechanistic and Physiological Models for Sustainable Food Cultivation

et al. 2021

View full text Add to dashboard Cite

Inefficiencies and imprecise input control in agriculture have caused devastating consequences to ecosystems. Urban controlled environment agriculture (CEA) is a proposed approach to mitigate the impacts of cultivation, but precise control of inputs (i.e., nutrient, water, etc.) is limited by the ability to monitor dynamic conditions. Current mechanistic and physiological plant growth models (MPMs) have not yet been unified and have uncovered knowledge gaps of the complex interplay among control variables. Moreover, because of their specificity, MPMs are of limited utility when extended to additional plant species or environmental conditions. Simultaneously, although machine learning (ML) can uncover latent interactions across conditions, phenotyping bottlenecks have hindered successful application. To bridge these gaps, we propose an integrative approach whereby MPMs are used to construct the foundations of ML algorithms, reducing data requirements and costs, and ML is used to elucidate parameters and causal inference in MPM. This review highlights research about control and automation in CEA, synthesizing literature into a framework whereby ML, MPM, and biofeedback inform what we call dynamically controlled environment agriculture (DCEA). We highlight synergistic characteristics of MPM and ML to illustrate that a DCEA framework could contribute to urban resilience, human health, and optimized productivity and nutritional content.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: 33mentioning

confidence: 99%

Dynamically Controlled Environment Agriculture: Integrating Machine Learning and Mechanistic and Physiological Models for Sustainable Food Cultivation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Such studies also have targeted only a single plant species to automate conventional farming and targeted only harvesting operations. Various image recognition technologies for automatic harvesting have been proposed for the farm environment and crops [23][24][25][26][27][28][29][30][31]. However, most of these technologies are targeted at conventional farming methods in which a single species is grown, thus lowering the recognition rate in environments where a variety of plants exist in small areas.…”

Section: Relevant Researchmentioning

confidence: 99%

Agricultural Robot under Solar Panels for Sowing, Pruning, and Harvesting in a Synecoculture Environment

et al. 2022

View full text Add to dashboard Cite

Currently, an agricultural method called SynecocultureTM has been receiving attention as a means for multiple crop production and recovering from environmental degradation; it helps in regreening the environment and establishing an augmented ecosystem with high biodiversity. In this method, several types of plants are grown densely, and their management relies mainly on manual labor, since conventional agricultural machines and robots cannot be applied in complex vegetation. To improve work efficiency and boost regreening by scaling-up Synecoculture, we developed a robot that can sow, prune, and harvest in dense and diverse vegetation that grows under solar panels, towards the achievement of compatibility between food and energy production on a large scale. We adopted a four-wheel mechanism with sufficient ability to move on uneven terrain, and a two orthogonal axes mechanism with adjusted tool positioning while performing management tasks. In the field experiment, the robot could move straight on shelving slopes and overcome obstacles, such as small steps and weeds, and succeeded in harvesting and weeding with human operation, using the tool maneuver mechanism based on the recognition of the field situation through camera image.

show abstract

“…LiDAR-based 3D object detection plays an indispensable role in 3D scene understanding with a wide range of applications such as autonomous driving (Deng et al, 2021; and robotics (Ahmed et al, 2018;Montes et al, 2020;. The emerging stream of 3D detection models enables accurate recognition at the cost of large-scale labeled point clouds, where 7-degree of freedom (DOF) 3D bounding boxes -consisting of a position, size, and orientation informationfor each object are annotated.…”

Section: Introductionmentioning

confidence: 99%

Exploring Active 3D Object Detection from a Generalization Perspective

Luo¹,

Chen²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

To alleviate the high annotation cost in LiDAR-based 3D object detection, active learning is a promising solution that learns to select only a small portion of unlabeled data to annotate, without compromising model performance. Our empirical study, however, suggests that mainstream uncertainty-based and diversitybased active learning policies are not effective when applied in the 3D detection task, as they fail to balance the trade-off between point cloud informativeness and box-level annotation costs. To overcome this limitation, we jointly investigate three novel criteria in our framework CRB for point cloud acquisitionlabel conciseness, feature representativeness and geometric balance, which hierarchically filters out the point clouds of redundant 3D bounding box labels, latent features and geometric characteristics (e.g., point cloud density) from the unlabeled sample pool and greedily selects informative ones with fewer objects to annotate. Our theoretical analysis demonstrates that the proposed criteria aligns the marginal distributions of the selected subset and the prior distributions of the unseen test set, and minimizes the upper bound of the generalization error. To validate the effectiveness and applicability of CRB, we conduct extensive experiments on the two benchmark 3D object detection datasets of KITTI and Waymo and examine both one-stage (i.e., SECOND) and two-stage 3D detectors (i.e., PV-RCNN). Experiments evidence that the proposed approach outperforms existing active learning strategies and achieves fully supervised performance requiring 1% and 8% annotations of bounding boxes and point clouds, respectively. Source code: https://github.com/Luoyadan/CRB-active-3Ddet.

show abstract

Real-time detection of broccoli crops in 3D point clouds for autonomous robotic harvesting

Cited by 11 publications

References 14 publications

Dynamically Controlled Environment Agriculture: Integrating Machine Learning and Mechanistic and Physiological Models for Sustainable Food Cultivation

Dynamically Controlled Environment Agriculture: Integrating Machine Learning and Mechanistic and Physiological Models for Sustainable Food Cultivation

Agricultural Robot under Solar Panels for Sowing, Pruning, and Harvesting in a Synecoculture Environment

Exploring Active 3D Object Detection from a Generalization Perspective

Contact Info

Product

Resources

About