Segmentation of Apple Point Clouds Based on Roi in RGB Images

YuanXi, Zhang; Tian, Yongchao; Zheng, Change; Zhao, Dong; Gao, Peng; Deng, Ke

doi:10.35633/inmateh-59-23

Cited by 7 publications

(2 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reference [26] employed the depth map of the Fotonic F80 depth camera from the detected sweet pepper regions and transformed the region to the 3D location of the mass center to obtain the locations of fruits. Reference [27] employed the point cloud of apple trees of Kinect v2 by fusing the RGB information and depth information, then segmenting the fruit regions by the point-cloud-segmentation method of the ROI in the RGB images and achieved a segmented purity of 96.7% and 96.2% for red and green apples. Reference [28] addressed the 3D pose estimation of peppers using Kinect Fusion technology from the Intel Realsense F200 RGBD camera to fit a superellipsoid from peppers' point clouds through a constrained non-linear least-squares optimization for the estimation of sweet pepper pose and grasp pose.…”

Section: D Fruit Localization and Approaching Direction Estimationmentioning

confidence: 99%

Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting

Feng

Qiu

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

Precise localization of occluded fruits is crucial and challenging for robotic harvesting in orchards. Occlusions from leaves, branches, and other fruits make the point cloud acquired from Red Green Blue Depth (RGBD) cameras incomplete. Moreover, an insufficient filling rate and noise on depth images of RGBD cameras usually happen in the shade from occlusions, leading to the distortion and fragmentation of the point cloud. These challenges bring difficulties to position locating and size estimation of fruit for robotic harvesting. In this paper, a novel 3D fruit localization method is proposed based on a deep learning segmentation network and a new frustum-based point-cloud-processing method. A one-stage deep learning segmentation network is presented to locate apple fruits on RGB images. With the outputs of masks and 2D bounding boxes, a 3D viewing frustum was constructed to estimate the depth of the fruit center. By the estimation of centroid coordinates, a position and size estimation approach is proposed for partially occluded fruits to determine the approaching pose for robotic grippers. Experiments in orchards were performed, and the results demonstrated the effectiveness of the proposed method. According to 300 testing samples, with the proposed method, the median error and mean error of fruits’ locations can be reduced by 59% and 43%, compared to the conventional method. Furthermore, the approaching direction vectors can be correctly estimated.

show abstract

Section: D Fruit Localization and Approaching Direction Estimationmentioning

confidence: 99%

Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting

Feng

Qiu

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…With the rapid development of information technology, agricultural mechanization is moving to the next new development stage, namely, agricultural precision (Hou, 2020). Scholars have conducted a series of studies on fruit and vegetable picking robots for apples Zhang et al, 2019), strawberries (Mejia et al, 2023;Xiong et al, 2019), kiwis (He et al, 2023;, citrus (Sun et al, 2023;Yin et al, 2023), and tomatoes (Feng et al, 2018;. Improving the picking efficiency and nondestructive picking of picking robots are hot research topics , and related scholars have proposed their own solutions.…”

Section: Introductionmentioning

confidence: 99%

Design and Experiment of Clamping-Pull-Off Apple Picking Robot

GUO,

FU,

CUI

et al. 2023

INMATEH

View full text Add to dashboard Cite

For the standardized apple orchards in China, which are mainly dwarfed and densely planted, firstly, according to the spatial distribution characteristics of fruits within the tree canopy, a clamping-pull-off apple picking robot was developed by analyzing the parameters of apple cultivation and picking methods, in order to replace the manual harvesting operation. Then, the D-H method was used to establish the kinematic Equations of the apple-picking robot, the forward and inverse kinematic calculations were carried out, and the Monte Carlo method was used to analyze the workspace of the robot. Through the robot picking task planning and the simulation of the trajectory of the robotic arm, the scheme of the robot's picking strategy was given, and MATLAB software was applied to simulate the motion trajectory as well as to verify the feasibility of the trajectory planning scheme and the picking strategy. Finally, an apple-picking test bed was set up, the corresponding picking control system program was designed, and 45 apples were selected for picking tests. The results showed that during the robot's picking process, the average time for picking each fruit was 7.59 seconds, the fruit recognition success rate was 86.67%, and the picking damage rate was 5.13%.

show abstract

Application of consumer RGB-D cameras for fruit detection and localization in field: A critical review

Gao

et al. 2020

Computers and Electronics in Agriculture

227

View full text Add to dashboard Cite

Segmentation of Apple Point Clouds Based on Roi in RGB Images

Cited by 7 publications

References 24 publications

Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting

Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting

Design and Experiment of Clamping-Pull-Off Apple Picking Robot

Application of consumer RGB-D cameras for fruit detection and localization in field: A critical review

Contact Info

Product

Resources

About