Rachis detection and three-dimensional localization of cut off point for vision-based banana robot

Wu, Fengyun; Duan, Jieli; Ai, Puye; Chen, Zhaoyi; Yang, Zhou; Zou, Xiangjun

doi:10.1016/j.compag.2022.107079

Cited by 84 publications

(43 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Fruit-picking robots are a class of agricultural machines that combine the advantages of the accuracy, efficiency, and characteristics of diverse sensors. They primarily perform automatic operations for crops in natural environments [1][2][3][4][5][6]. Among them, the deployment of machine vision systems and corresponding recognition algorithms allows them to efficiently complete many harvesting operations.…”

Section: Introductionmentioning

confidence: 99%

A Study on Long-Close Distance Coordination Control Strategy for Litchi Picking

Wang

Lin

et al. 2022

Agronomy

Self Cite

View full text Add to dashboard Cite

For the automated robotic picking of bunch-type fruit, the strategy is to roughly determine the location of the bunches, plan the picking route from a remote location, and then locate the picking point precisely at a more appropriate, closer location. The latter can reduce the amount of information to be processed and obtain more precise and detailed features, thus improving the accuracy of the vision system. In this study, a long–close distance coordination control strategy for a litchi picking robot was proposed based on an Intel Realsense D435i camera combined with a point cloud map collected by the camera. The YOLOv5 object detection network and DBSCAN point cloud clustering method were used to determine the location of bunch fruits at a long distance to then deduce the sequence of picking. After reaching the close-distance position, the Mask RCNN instance segmentation method was used to segment the more distinctive bifurcate stems in the field of view. By processing segmentation masks, a dual reference model of “Point + Line” was proposed, which guided picking by the robotic arm. Compared with existing studies, this strategy took into account the advantages and disadvantages of depth cameras. By experimenting with the complete process, the density-clustering approach in long distance was able to classify different bunches at a closer distance, while a success rate of 88.46% was achieved during fruit-bearing branch locating. This was an exploratory work that provided a theoretical and technical reference for future research on fruit-picking robots.

show abstract

Section: Introductionmentioning

confidence: 99%

A Study on Long-Close Distance Coordination Control Strategy for Litchi Picking

Wang

Lin

et al. 2022

Agronomy

Self Cite

View full text Add to dashboard Cite

show abstract

“…Robotic harvesting technology has been actively explored for decades as a potential solution to address the increasingly severe labour uncertainty. From the original idea proposed by Schertz and Brown [ 1 ] to the fully integrated robotic harvesting systems booming in the past ten years [ 2 , 3 ], significant advancements have been made in machine vision [ 4 , 5 , 6 , 7 ], path planning [ 8 ], fruit extraction [ 9 ], and grasping [ 10 ]. However, these achievements in robotic harvesting have yet to secure the level of productivity required to see widespread adoption.…”

Section: Introductionmentioning

confidence: 99%

Learning-Based Slip Detection for Robotic Fruit Grasping and Manipulation under Leaf Interference

Zhou

Xiao

Kang

et al. 2022

Sensors

View full text Add to dashboard Cite

Robotic harvesting research has seen significant achievements in the past decade, with breakthroughs being made in machine vision, robot manipulation, autonomous navigation and mapping. However, the missing capability of obstacle handling during the grasping process has severely reduced harvest success rate and limited the overall performance of robotic harvesting. This work focuses on leaf interference caused slip detection and handling, where solutions to robotic grasping in an unstructured environment are proposed. Through analysis of the motion and force of fruit grasping under leaf interference, the connection between object slip caused by leaf interference and inadequate harvest performance is identified for the first time in the literature. A learning-based perception and manipulation method is proposed to detect slip that causes problematic grasps of objects, allowing the robot to implement timely reaction. Our results indicate that the proposed algorithm detects grasp slip with an accuracy of 94%. The proposed sensing-based manipulation demonstrated great potential in robotic fruit harvesting, and could be extended to other pick-place applications.

show abstract

“…As the key of automatic picking, visual recognition system can determine the efficiency of machine picking. The vision system aiming at accurate location and segmentation has important research significance for various automatic agricultural applications (Jana et al, 2017 ; Fu et al, 2020 ; Tang et al, 2020 ; Chen et al, 2021 ; Wu et al, 2022 ). Compared with manual picking, automatic machine picking can improve picking efficiency and save labor cost.…”

Section: Introductionmentioning

confidence: 99%

Accurate segmentation of green fruit based on optimized mask RCNN application in complex orchard

et al. 2022

View full text Add to dashboard Cite

Fruit and vegetable picking robots are affected by the complex orchard environment, resulting in poor recognition and segmentation of target fruits by the vision system. The orchard environment is complex and changeable. For example, the change of light intensity will lead to the unclear surface characteristics of the target fruit; the target fruits are easy to overlap with each other and blocked by branches and leaves, which makes the shape of the fruits incomplete and difficult to accurately identify and segment one by one. Aiming at various difficulties in complex orchard environment, a two-stage instance segmentation method based on the optimized mask region convolutional neural network (mask RCNN) was proposed. The new model proposed to apply the lightweight backbone network MobileNetv3, which not only speeds up the model but also greatly improves the accuracy of the model and meets the storage resource requirements of the mobile robot. To further improve the segmentation quality of the model, the boundary patch refinement (BPR) post-processing module is added to the new model to optimize the rough mask boundaries of the model output to reduce the error pixels. The new model has a high-precision recognition rate and an efficient segmentation strategy, which improves the robustness and stability of the model. This study validates the effect of the new model using the persimmon dataset. The optimized mask RCNN achieved mean average precision (mAP) and mean average recall (mAR) of 76.3 and 81.1%, respectively, which are 3.1 and 3.7% improvement over the baseline mask RCNN, respectively. The new model is experimentally proven to bring higher accuracy and segmentation quality and can be widely deployed in smart agriculture.

show abstract

Rachis detection and three-dimensional localization of cut off point for vision-based banana robot

Cited by 84 publications

References 24 publications

A Study on Long-Close Distance Coordination Control Strategy for Litchi Picking

A Study on Long-Close Distance Coordination Control Strategy for Litchi Picking

Learning-Based Slip Detection for Robotic Fruit Grasping and Manipulation under Leaf Interference

Accurate segmentation of green fruit based on optimized mask RCNN application in complex orchard

Contact Info

Product

Resources

About