Development of a sweet pepper harvesting robot

Abstract

This paper presents the development, testing and validation of SWEEPER, a robot for harvesting sweet pepper fruit in greenhouses. The robotic system includes a six degrees of freedom industrial arm equipped with a specially designed end effector, RGB-D camera, high-end computer with graphics processing unit, programmable logic controllers, other electronic equipment, and a small container to store harvested fruit. All is mounted on a cart that autonomously drives on pipe rails and concrete floor in the end-user environment. The overall operation of the harvesting robot is described along with details of the algorithms for fruit detection and localization, grasp pose estimation, and motion control. The main contributions of this paper are the integrated system design and its validation and extensive field testing in a commercial greenhouse for different varieties and growing conditions. A total of 262 fruits were involved in a 4-week long testing period. The average cycle time to harvest a fruit was 24 s. Logistics took approximately 50% of this time (7.8 s for discharge of fruit and 4.7 s for platform movements). Laboratory experiments have proven that the cycle time can be reduced to 15 s by running the robot manipulator at a higher speed. The harvest success rates were 61% for the best fit crop conditions and 18% in current crop conditions. This reveals the importance of finding the best fit crop conditions and crop varieties for successful robotic harvesting. The SWEEPER robot is the first sweet pepper harvesting robot to demonstrate this kind of performance in a commercial greenhouse.

Heuristic Evaluation of the User Interface for a Semi-Autonomous Agricultural Robot Sprayer

“…Due to technological advancements in recent years, robotics has begun to play a major role in our daily lives (Arad et al, 2020). Automation in agriculture, mechanization and agricultural engineering, has been a major force for increased productivity in the 20th century (McNulty and Grace, 2009).…”

Design and evaluation of a modular robotic plum harvesting system utilizing soft components

“…System implementations of robotic harvesters using modern techniques are presented by Arad et al (2020) for sweet peppers and by Xiong, Peng, Grimstad, From, and Isler (2019) for strawberries. Many additional research efforts have been directed towards autonomous harvesting in protected cultivation systems (Tanigaki, Fujiura, Akase, & Imagawa, 2008; van Henten et al, 2002, 2013).…”

“…Synthetic image generation is explored by Barth, Ijsselmuiden, Hemming, and van Henten (2018) as a means of overcoming the need to gather large training datasets. A deep network is combined with morphological and color thresholding to yield high frame rates while detecting sweet peppers and fruit in Arad et al (2020), though this operates on yellow fruit. Lighting control is effective for greenhouse environments through filtering and flash‐no‐flash image sequences, as demonstrated in Arad et al (2019).…”

“…A commercial deformable‐finger end‐effector design is used by Eizicovits, van Tuijl, Berman, and Edan (2016), where simulation tools are used to map grasp position tolerances to sensing requirements. Crops such as sweet peppers require stem cutting which has been addressed in combination with suction (Bac & Hemming, 2017; Lehnert, English, McCool, Tow, & Perez, 2017) and catching (Arad et al, 2020) type grippers.…”

Controlled Lighting and Illumination-Independent Target Detection for Real-Time Cost-Efficient Applications. The Case Study of Sweet Pepper Robotic Harvesting

“…This simple calibration approach was designed to require only a small number of “training” images (3) and can be performed quickly—thus facilitating rapid adaptation to new environments (e.g., different greenhouses, growing conditions, pepper varieties). This advantage was utilized during the SWEEPER pepper harvesting robot development [29] in order to adapt the algorithm to an artificial plant model used for indoor testing (see Figure 4).…”