An automated apple harvesting robot—From system design to field evaluation

Zhang, Kaixiang; Lammers, Kyle; Chu, Pengyu; Li, Zhaojian; Lu, Renfu

doi:10.1002/rob.22268

Cited by 20 publications

(6 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comparing the results with other recent field-tested fruit-harvesting robots, we find that Yin et al [23] tested a citrus-harvesting robot in the actual field with a success rate of 87.2% and a pick up time of 10.9 s. Furthermore, Zhang et al [47] harvested apples in the field using a four-degree-of-freedom robot using the CNN algorithm for fruit detection having a success rate of 82.4% and a pick time of 6 s. The fruit detection accuracy was 90%. Moreover, Hu et al [48] also tested a citrus-harvesting robot with a success rate of 90% and a pick up time of 15 s.…”

Section: Discussionmentioning

confidence: 55%

See 1 more Smart Citation

In-Depth Evaluation of Automated Fruit Harvesting in Unstructured Environment for Improved Robot Design

Zeeshan,

Aized,

Riaz

2024

Machines

View full text Add to dashboard Cite

Using modern machines like robots comes with its set of challenges when encountered with unstructured scenarios like occlusion, shadows, poor illumination, and other environmental factors. Hence, it is essential to consider these factors while designing harvesting robots. Fruit harvesting robots are modern automatic machines that have the ability to improve productivity and replace labor for repetitive and laborious harvesting tasks. Therefore, the aim of this paper is to design an improved orange-harvesting robot for a real-time unstructured environment of orchards, mainly focusing on improved efficiency in occlusion and varying illumination. The article distinguishes itself with not only an efficient structural design but also the use of an enhanced convolutional neural network, methodologically designed and fine-tuned on a dataset tailored for oranges integrated with position visual servoing control system. Enhanced motion planning uses an improved rapidly exploring random tree star algorithm that ensures the optimized path for every robot activity. Moreover, the proposed machine design is rigorously tested to validate the performance of the fruit harvesting robot. The unique aspect of this paper is the in-depth evaluation of robots to test five areas of performance that include not only the accurate detection of the fruit, time of fruit picking, and success rate of fruit picking, but also the damage rate of fruit picked as well as the consistency rate of the robot picking in varying illumination and occlusion. The results are then analyzed and compared with the performance of a previous design of fruit harvesting robot. The study ensures improved results in most aspects of the design for performance in an unstructured environment.

show abstract

Section: Discussionmentioning

confidence: 55%

Section: Discussionmentioning

confidence: 65%

In-Depth Evaluation of Automated Fruit Harvesting in Unstructured Environment for Improved Robot Design

Zeeshan,

Aized,

Riaz

2024

Machines

View full text Add to dashboard Cite

show abstract

“…Furthermore, an active laser scanning scheme (ALACS) was introduced with the help of an FLIR camera to improve localization accuracy, and laser scanning was used to recalculate the 3D position of the apple by comparing the depth frame obtained from the RealSense RGB-D camera. Robust calibration was conducted based on the random sample consensus method for calibrating the model parameters related to collected data [ 44 , 45 ]. The sensor fusion method that has already been used requires complex calibration methods, especially when fusion was conducted with LiDAR [ 43 ] or high processing power based on the complexity of the algorithm, which requires high computational power in applications with robotic systems.…”

Section: Related Workmentioning

confidence: 99%

3D Camera and Single-Point Laser Sensor Integration for Apple Localization in Spindle-Type Orchard Systems

Abeyrathna,

Nakaguchi,

Liu

et al. 2024

Sensors

View full text Add to dashboard Cite

Accurate localization of apples is the key factor that determines a successful harvesting cycle in the automation of apple harvesting for unmanned operations. In this regard, accurate depth sensing or positional information of apples is required for harvesting apples based on robotic systems, which is challenging in outdoor environments because of uneven light variations when using 3D cameras for the localization of apples. Therefore, this research attempted to overcome the effect of light variations for the 3D cameras during outdoor apple harvesting operations. Thus, integrated single-point laser sensors for the localization of apples using a state-of-the-art model, the EfficientDet object detection algorithm with an mAP@0.5 of 0.775 were used in this study. In the experiments, a RealSense D455f RGB-D camera was integrated with a single-point laser ranging sensor utilized to obtain precise apple localization coordinates for implementation in a harvesting robot. The single-point laser range sensor was attached to two servo motors capable of moving the center position of the detected apples based on the detection ID generated by the DeepSORT (online real-time tracking) algorithm. The experiments were conducted under indoor and outdoor conditions in a spindle-type apple orchard artificial architecture by mounting the combined sensor system behind a four-wheel tractor. The localization coordinates were compared between the RGB-D camera depth values and the combined sensor system under different light conditions. The results show that the root-mean-square error (RMSE) values of the RGB-D camera depth and integrated sensor mechanism varied from 3.91 to 8.36 cm and from 1.62 to 2.13 cm under 476~600 lx to 1023~1100 × 100 lx light conditions, respectively. The integrated sensor system can be used for an apple harvesting robotic manipulator with a positional accuracy of ±2 cm, except for some apples that were occluded due to leaves and branches. Further research will be carried out using changes in the position of the integrated system for recognition of the affected apples for harvesting operations.

show abstract

“…The control systems, navigation algorithm, and objection detection algorithms utilized in various agricultural and non-agricultural robots [5][6][7][8][9][10][11][12][13] can also be adapted for cotton harvesting robots. Several research articles have focused on distinct sub-components of a robotic cotton harvesting system, such as the development of a cotton boll detection model [14][15][16][17][18][19], navigation and path planning algorithms [20][21][22][23][24][25], and end-effector designs [26][27][28][29][30].…”

Section: Introductionmentioning

confidence: 99%

Robotic Multi-Boll Cotton Harvester System Integration and Performance Evaluation

Thapa,

Rains,

Porter

et al. 2024

AgriEngineering

View full text Add to dashboard Cite

Several studies on robotic cotton harvesters have designed their end-effectors and harvesting algorithms based on the approach of harvesting a single cotton boll at a time. These robotic cotton harvesting systems often have slow harvesting times per boll due to limited computational speed and the extended time taken by actuators to approach and retract for picking individual cotton bolls. This study modified the design of the previous version of the end-effector with the aim of improving the picking ratio and picking time per boll. This study designed and fabricated a pullback reel to pull the cotton plants backward while the rover harvested and moved down the row. Additionally, a YOLOv4 cotton detection model and hierarchical agglomerative clustering algorithm were implemented to detect cotton bolls and cluster them. A harvesting algorithm was then developed to harvest the cotton bolls in clusters. The modified end-effector, pullback reel, vacuum conveying system, cotton detection model, clustering algorithm, and straight-line path planning algorithm were integrated into a small red rover, and both lab and field tests were conducted. In lab tests, the robot achieved a picking ratio of 57.1% with an average picking time of 2.5 s per boll. In field tests, picking ratio was 56.0%, and it took an average of 3.0 s per boll. Although there was no improvement in the lab setting over the previous design, the robot’s field performance was significantly better, with a 16% higher picking ratio and a 46% reduction in picking time per boll compared to the previous end-effector version tested in 2022.

show abstract

An automated apple harvesting robot—From system design to field evaluation

Cited by 20 publications

References 40 publications

In-Depth Evaluation of Automated Fruit Harvesting in Unstructured Environment for Improved Robot Design

In-Depth Evaluation of Automated Fruit Harvesting in Unstructured Environment for Improved Robot Design

3D Camera and Single-Point Laser Sensor Integration for Apple Localization in Spindle-Type Orchard Systems

Robotic Multi-Boll Cotton Harvester System Integration and Performance Evaluation

Contact Info

Product

Resources

About