Peduncle Detection of Sweet Pepper for Autonomous Crop Harvesting—Combined Color and 3-D Information

Abstract

This paper presents a 3D visual detection method for the challenging task of detecting peduncles of sweet peppers (Capsicum annuum) in the field. Cutting the peduncle cleanly is one of the most difficult stages of the harvesting process, where the peduncle is the part of the crop that attaches it to the main stem of the plant. Accurate peduncle detection in 3D space is therefore a vital step in reliable autonomous harvesting of sweet peppers, as this can lead to precise cutting while avoiding damage to the surrounding plant. This paper makes use of both colour and geometry information acquired from an RGB-D sensor and utilises a supervised-learning approach for the peduncle detection task. The performance of the proposed method is demonstrated and evaluated using qualitative and quantitative results (the Area-Under-the-Curve (AUC) of the detection precision-recall curve). We are able to achieve an AUC of 0.71 for peduncle detection on field-grown sweet peppers. We release a set of manually annotated 3D sweet pepper and peduncle images to assist the research community in performing further research on this topic.

Towards Automated Strawberry Harvesting: Identifying the Picking Point

“…While colour is a good guide, in some situations we will need more. Our current thought is that supervised machine learning, along the lines described in [12], is one way to try and improve our approach.…”

“…This point can then be grabbed with a manipulator without damaging the fruit. This is the preferred method of harvesting many soft fruits as well as some other crops such as peppers [12].…”

Towards Active Robotic Vision in Agriculture: A Deep Learning Approach to Visual Servoing in Occluded and Unstructured Protected Cropping Environments

“…This would require depth measurement per pixel via stereo-matching which is computationally expensive, and when done with a short baseline between imaging sensors, such as the case with 3DMTS, often yields depth estimates with high uncertainty (Gongal et al, 2015). Tanigaki et al (2008) and Sa et al (2017) both used a depth augmented imaging sensor to obtain a 3D point cloud of the scene around the target fruit (cherries and sweet peppers, respectively) which was coupled with image segmentation techniques to identify regions attributed to fruit, stems and leaves. In both cases, the geometry was exploited to locate cutting sites for harvesting.…”

3D‐vision based detection, localization, and sizing of broccoli heads in the field

“…used combined information from RGB and depth for detecting sweet peppers, by detecting highlights in the image planes on registered RGB‐D images to identify fruit regions and classifying peppers based on surface normal distribution and 3D object symmetry. Sa et al . use an RGB‐D sensor to detect sweet pepper peduncles to facilitate robotic harvesting.…”

“…Barnea et al 10 used combined information from RGB and depth for detecting sweet peppers, by detecting highlights in the image planes on registered RGB-D images to identify fruit regions and classifying peppers based on surface normal distribution and 3D object symmetry. Sa et al 11 use an RGB-D sensor to detect sweet pepper peduncles to facilitate robotic harvesting. The cucumber harvesting robot by van Henten et al 12 employs high resolution CCD cameras for detection of crops in greenhouses and 3D data for localization.…”