Fast Recognition and Location of Target Fruit Based on Depth Information

Ye, Tian; Duan, Huichuan; Luo, Rong; Zhang, Yan; Jia, Weikuan; Lian, Jian; Zheng, Yuanjie; Ruan, Chengzhi; Li, Chengjiang

doi:10.1109/access.2019.2955566

Cited by 39 publications

(17 citation statements)

References 73 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Scan the maximum radius of superpixel area where the center of apple is located and fit the target fruit, it is shown in Figure 5. 22 Li used binocular stereo vision technology to obtain the three-dimensional position of the apple and improved the matching accuracy of the two pictures by reducing the search range and the range error was 5%. 23 However, the actual harvesting of the environment is far more complicated than the experimental conditions, the time consumption will obviously increase, and the real-time performance will also be significantly worse.…”

Section: Precise Recognition For Target Fruitmentioning

confidence: 99%

Apple harvesting robot under information technology: A review

Jia

Yan

Lian

et al. 2020

International Journal of Advanced Robotic Systems

Self Cite

103

View full text Add to dashboard Cite

It has been more than 30 years since the French pioneered the research of the apple harvesting robot, and with the joint efforts of scholars all around the world, a variety of prototypes have been developed. However, the existing apple harvesting robot prototype is still in the experimental research stage because of its low harvesting efficiency. With the help of information technology, the related research has ushered in a milestone development, and it is full of opportunities and challenges for apple harvesting robotic researchers. In this article, it briefly introduced the development history, structure, and composition of apple harvesting robots and the operation process, which makes readers have a clear understanding of apple harvesting robot and its harvesting principle. Then systematically summarizing the research results of apple harvesting robots both at domestic and at foreign, we carried out in following three aspects: rapid and accurate recognition and positioning of target fruit, all-weather operation mode, and application of intelligent computing theory in apple harvesting robots, and it analyzes the research progress of apple harvesting robot in detail. The results show that improving the harvesting efficiency is the key and hot spot for the research on apple harvesting robots. Under the impetus of information technology, how to achieve fast and accurate recognition of the fruits of multienvironment and multiple information, obtain reasonable path planning, and further optimization of control strategies are all important research directions.

show abstract

Section: Precise Recognition For Target Fruitmentioning

confidence: 99%

Apple harvesting robot under information technology: A review

Jia

Yan

Lian

et al. 2020

International Journal of Advanced Robotic Systems

Self Cite

103

View full text Add to dashboard Cite

show abstract

“…The gradient information is obtained from the depth image, and the segmentation algorithm is introduced into the red–green–blue (RGB) image. Finally, the center and maximum radius of the target fruit was scanned to fit the contour size of the target fruit, so as to realize apple recognition and location (Tian et al, 2019 ). Wu proposed a fruit point cloud segmentation algorithm integrating color and three-dimensional geometric features.…”

Section: Introductionmentioning

confidence: 99%

Accurate segmentation of green fruit based on optimized mask RCNN application in complex orchard

et al. 2022

Self Cite

View full text Add to dashboard Cite

Fruit and vegetable picking robots are affected by the complex orchard environment, resulting in poor recognition and segmentation of target fruits by the vision system. The orchard environment is complex and changeable. For example, the change of light intensity will lead to the unclear surface characteristics of the target fruit; the target fruits are easy to overlap with each other and blocked by branches and leaves, which makes the shape of the fruits incomplete and difficult to accurately identify and segment one by one. Aiming at various difficulties in complex orchard environment, a two-stage instance segmentation method based on the optimized mask region convolutional neural network (mask RCNN) was proposed. The new model proposed to apply the lightweight backbone network MobileNetv3, which not only speeds up the model but also greatly improves the accuracy of the model and meets the storage resource requirements of the mobile robot. To further improve the segmentation quality of the model, the boundary patch refinement (BPR) post-processing module is added to the new model to optimize the rough mask boundaries of the model output to reduce the error pixels. The new model has a high-precision recognition rate and an efficient segmentation strategy, which improves the robustness and stability of the model. This study validates the effect of the new model using the persimmon dataset. The optimized mask RCNN achieved mean average precision (mAP) and mean average recall (mAR) of 76.3 and 81.1%, respectively, which are 3.1 and 3.7% improvement over the baseline mask RCNN, respectively. The new model is experimentally proven to bring higher accuracy and segmentation quality and can be widely deployed in smart agriculture.

show abstract

“…The recognition accuracy of green apples reached 88%. Tian [11] proposed a target fruit localization method based on depth image, and located the center and radius of the apple circle respectively through depth image and its corresponding RGB spatial information, so as to fit the target area. Li [12] combined applying saliency detection and Gaussian curve fitting algorithm, a novel algorithm is used to detect green apples in natural scenes, the experimental results indicated that it was effective and feasible.…”

Section: Introduction mentioning

confidence: 99%

Novel green-fruit detection algorithm based on D2D framework

Wei

Liu

Ullah

et al. 2022

International Journal of Agricultural and Biological Engineering

Self Cite

View full text Add to dashboard Cite

In the complex orchard environment, the efficient and accurate detection of object fruit is the basic requirement to realize the orchard yield measurement and automatic harvesting. Sometimes it is hard to differentiate between the object fruits and the background because of the similar color, and it is challenging due to the ambient light and camera angle by which the photos have been taken. These problems make it hard to detect green fruits in orchard environments. In this study, a two-stage dense to detection framework (D2D) was proposed to detect green fruits in orchard environments. The proposed model was based on multi-scale feature extraction of target fruit by using feature pyramid networks MobileNetV2 +FPN structure and generated region proposal of target fruit by using Region Proposal Network (RPN) structure. In the regression branch, the offset of each local feature was calculated, and the positive and negative samples of the region proposals were predicted by a binary mask prediction to reduce the interference of the background to the prediction box. In the classification branch, features were extracted from each sub-region of the region proposal, and features with distinguishing information were obtained through adaptive weighted pooling to achieve accurate classification. The new proposed model adopted an anchor-free frame design, which improves the generalization ability, makes the model more robust, and reduces the storage requirements. The experimental results of persimmon and green apple datasets show that the new model has the best detection performance, which can provide theoretical reference for other green object detection.

show abstract

Fast Recognition and Location of Target Fruit Based on Depth Information

Cited by 39 publications

References 73 publications

Apple harvesting robot under information technology: A review

Apple harvesting robot under information technology: A review

Accurate segmentation of green fruit based on optimized mask RCNN application in complex orchard

Novel green-fruit detection algorithm based on D2D framework

Contact Info

Product

Resources

About