A lab-customized autonomous humanoid apple harvesting robot

Yu, Xiaojun; Zhao, Fan; Wang, Xingduo; Wan, Hao; Wang, Pengbo; Zeng, Xilei; Feng, Jian

doi:10.1016/j.compeleceng.2021.107459

Cited by 38 publications

(24 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a large apple producing country, China's apple production accounts for half of the global production. With population mobility and changes in land policies, the mechanization and intelligence of agricultural production are the main trends in future development [1]. Until now, the domestic apple-picking method was based on manual picking.…”

Section: Introductionmentioning

confidence: 99%

[Retracted] Fast Recognition Method for Multiple Apple Targets in Complex Occlusion Environment Based on Improved YOLOv5

2023

View full text Add to dashboard Cite

The mechanization and intelligentization of the production process are the main trends in research and development of agricultural products. The realization of an unmanned and automated picking process is also one of the main research hotspots in China’s agricultural product engineering technology field in recent years. The development of automated apple-picking robot is directly related to imaging research, and its key technology is to use algorithms to realize apple identification and positioning. Aiming at the problem of false detection and missed detection of densely occluded targets and small targets by apple picking robots under different lighting conditions, two different apple recognition algorithms are selected based on the apple shape features to study the traditional machine learning algorithm: histogram of oriented gradients + support vector machine (HOG + SVM) and a fast recognition method for multiple apple targets in a complex occlusion environment based on improved You-Only-Look-Once-v5 (YOLOv5). The first is the improvement of the CSP structure in the network. Using parameter reconstruction, the convolutional layer (Conv) and the batch normalization (BN) layer in the CBL (Conv + BN + Leaky_relu activation function) module are fused into a batch-normalized convolutional layer Conv_B. Subsequently, the CA (coordinate attention) mechanism module is embedded into different network layers in the improved designed backbone network to enhance the expressive ability of the features in the backbone network to better extract the features of different apple targets. Finally, for some targets with overlapping occlusions, the loss function is fine-tuned to improve the model’s ability to recognize occluded targets. By comparing the recognition effects of HOG + SVM, Faster RCNN, YOLOv6, and baseline YOLOv5 on the test set under complex occlusion scenarios, the F 1 value of this method was increased by 13.47%, 6.01%, 1.26%, and 3.63%, respectively, and the F 1 value of this method was increased by 19.36%, 13.07%, 1.61%, and 4.27%, respectively, under different illumination angles. The average image recognition time was 0.27 s faster than that of HOG + SVM, 0.229 s faster than that of Faster RCNN, and 0.006 s faster than that of YOLOv6. The method is expected to provide a theoretical basis for apple-picking robots to choose a pertinent image recognition algorithm during operation.

show abstract

Section: Introductionmentioning

confidence: 99%

[Retracted] Fast Recognition Method for Multiple Apple Targets in Complex Occlusion Environment Based on Improved YOLOv5

2023

View full text Add to dashboard Cite

show abstract

“…Pepper RGB-D [10,21], Binocular camera [22] Fruit color [21], 3D point cloud [21], spatial coordinates [21], fruit stalk posture [21], plant main stem morphology [10] Detection range 200~600 mm [22], Height range 1000 mm [23] Strawberry Laser ranging sensor [24], 3 CCD cameras [25], binocular camera [26], RGB-D [5], infrared sensor [5] Fruit color [25], position [26], and stem posture [5] Detection range 200~700 mm [26,27], Width range 350~670 mm [26,27], Height range 200~300 mm [28] Tomato Photoelectric sensor [29], binocular camera [30], laser sensor [9], RGB-D [31] Fruit color [9], size [9] and position [30], fruit stalk posture [31] Detection range 400~1000 mm [32], Height range 600 mm [33] Apple Binocular camera [34], RGB-D [35] Fruit color [34,35], size [34,35] and position [34,35] Detection range 1000~2000 mm [36], Height range 1000~1500 mm [7] Citrus Binocular camera [37], RGB-D …”

Section: Object Fruit Sensor Visual Information View Fieldmentioning

confidence: 99%

Advance of Target Visual Information Acquisition Technology for Fresh Fruit Robotic Harvesting: A Review

Feng

et al. 2022

Agronomy

View full text Add to dashboard Cite

In view of the continuous increase in labor costs for complex picking tasks, there is an urgent demand for intelligent harvesting robots in the global fresh fruit cultivation industry. Fruit visual information is essential to guide robotic harvesting. However, obtaining accurate visual information about the target is critical in complex agricultural environments. The main challenges include the image color distortion under changeable natural light, occlusions from the interlaced plant organs (stems, leaves, and fruits), and the picking point location on fruits with variable shapes and poses. On top of summarizing the current status of typical fresh fruit harvesting robots, this paper outlined the state-of-the-art advance of visual information acquisition technology, including image acquisition in the natural environment, fruit recognition from the complex backgrounds, target stereo locating and measurement, and fruit search among the plants. It then analyzed existing problems and raised future potential research trends from two aspects, multiple images fusion and self-improving algorithm model.

show abstract

“…12 (g) Apple harvesting robot with Universal robot (UR5) manipulator and RGB-D vision camera. 11 (h) Apple harvesting robot with Dynamixel SDK hardware control software 185 (i) Humanoid harvesting robots in Apple 186 and Grape 55 detection.…”

Section: Fruit Harvesting Robotsmentioning

confidence: 99%

Selective fruit harvesting: Research, trends and developments towards fruit detection and localization – A review

Kumar

Mohan

2022

Proceedings of the Institution of Mechanical Engineers, Part C:

View full text Add to dashboard Cite

Progressive application of multidisciplinary research and development pushes the evolution of automation in many subsectors of agriculture to increase productivity, economic growth and environmental preservation with the help of robotics and artificial intelligence. Fruit harvesting robots have been developed mainly to provide support in the field for limited labour resources, to enable selective harvesting, to improve the efficiency and to preserve the quality of fruits. Even a small delay in harvesting can cause a maximum impact to the quality of the fruit. Selective fruit harvesting is an integration of different subcomponents. This paper provides a brief analysis of the techniques in selective fruit harvesting for the past 6 years starting from 2017 to 2022, associated principles, limitations and directions for future challenges. The first subcomponent is the vision system, it captures the information about the fruit in a tree canopy to perform efficient 2D and 3D localization. Hence getting accurate information from the vision system is more essential even in a complex agricultural environment. The detection of fruit from the vision system is normally done with two major methods such as traditional image processing and deep learning approaches. The result shows that the traditional methods provide high efficiency but the colour similarity, complex backgrounds and lightning conditions often makes failure in detection. Shortage of standard dataset and high-powered processing devices hinders the development of deep learning algorithms. Also, the usage of large data sets reduces the training speed even in pre-trained networks. For fruit grasping and detachment, the detection of plucking point is more essential to preserve the quality and for further storage. The elaborate description about the framework, limitations in current sensing, fruit and picking point detection algorithms provides guidelines to the researchers in building a fully automated robotic system to increase the processing speed and production rate.

show abstract

A lab-customized autonomous humanoid apple harvesting robot

Cited by 38 publications

References 17 publications

[Retracted] Fast Recognition Method for Multiple Apple Targets in Complex Occlusion Environment Based on Improved YOLOv5

[Retracted] Fast Recognition Method for Multiple Apple Targets in Complex Occlusion Environment Based on Improved YOLOv5

Advance of Target Visual Information Acquisition Technology for Fresh Fruit Robotic Harvesting: A Review

Selective fruit harvesting: Research, trends and developments towards fruit detection and localization – A review

Contact Info

Product

Resources

About