Algorithm Design and Integration for a Robotic Apple Harvesting System

Abstract

Due to labor shortage and rising labor cost for the apple industry, there is an urgent need for the development of robotic systems to efficiently and autonomously harvest apples. In this paper, we present a system overview and algorithm design of our recently developed robotic apple harvester prototype. Our robotic system is enabled by the close integration of several core modules, including calibration, visual perception, planning, and control. This paper covers the main methods and advancements in robust extrinsic parameter calibration, deep learning-based multi-view fruit detection and localization, unified picking and dropping planning, and dexterous manipulation control. Indoor and field experiments were conducted to evaluate the performance of the developed system, which achieved an average picking rate of 3.6 seconds per apple. This is a significant improvement over other reported apple harvesting robots with a picking rate in the range of 7-10 seconds per apple. The current prototype shows promising performance towards further development of efficient and automated apple harvesting technology. Finally, limitations of the current system and future work are discussed.This work is partly supported by the USDA-ARS inhouse project 5050-43640-003-00D.

Advance of Target Visual Information Acquisition Technology for Fresh Fruit Robotic Harvesting: A Review

“…Fu [46] set the foreground LED light to highlight the contour boundary of overlapping targets and reduce background interference for acquiring kiwi fruit images at night, and the detection accuracy was 88.3%. Zhang [47] addressed the problem of shadow and high brightness on the fruit surface through fusing the multi-view images, and the apple recognition accuracy was improved from 90.5% to 93.2%. To overcome the color distortion of the sweet pepper plant image under the background with intensive radiation, Arab [48] integrated the image under the condition of natural light and artificial light by the Flash-No-Flash (FNF) controlled illumination unit (Figure 3), and the fruit recognition accuracy was improved by 4%.…”