Dynamic thresholding algorithm for robotic apple detection

Zemmour, Elie; Kurtser, Polina; Edan, Yael

doi:10.1109/icarsc.2017.7964082

Cited by 6 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The current paper advances previous research [30] with several new contributions: (1) a new parameter tuning procedure developed to best-fit the parameters to the specific database; (2) the application and evaluation of the adaptive thresholding algorithm for different color spaces; (3) application of the algorithm to different types of fruits along with intensive evaluation and sensitivity analyses; (4) comparing the contribution of the new developments (Items 1–2) to previous developments.…”

Section: Introductionsupporting

confidence: 79%

“…Research was done to identify the PDF function of the data distributions of each database through a χ2 goodness of fit test. However, since these tests did not reveal significant results [54], the thresholds were selected as follows: T1 and T2 were chosen so that 15% of the data would be categorized as low, 15% as high, and 70% as medium.…”

Section: Algorithmmentioning

confidence: 99%

“…A set of three thresholds was determined for each region of the image according to its lighting setting. Preliminary results of the same algorithm for an apple detection problem have been previously presented [30].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automatic Parameter Tuning for Adaptive Thresholding in Fruit Detection

Zemmour

Kurtser

Edan

2019

Sensors

Self Cite

View full text Add to dashboard Cite

This paper presents an automatic parameter tuning procedure specially developed for a dynamic adaptive thresholding algorithm for fruit detection. One of the major algorithm strengths is its high detection performances using a small set of training images. The algorithm enables robust detection in highly-variable lighting conditions. The image is dynamically split into variably-sized regions, where each region has approximately homogeneous lighting conditions. Nine thresholds were selected to accommodate three different illumination levels for three different dimensions in four color spaces: RGB, HSI, LAB, and NDI. Each color space uses a different method to represent a pixel in an image: RGB (Red, Green, Blue), HSI (Hue, Saturation, Intensity), LAB (Lightness, Green to Red and Blue to Yellow) and NDI (Normalized Difference Index, which represents the normal difference between the RGB color dimensions). The thresholds were selected by quantifying the required relation between the true positive rate and false positive rate. A tuning process was developed to determine the best fit values of the algorithm parameters to enable easy adaption to different kinds of fruits (shapes, colors) and environments (illumination conditions). Extensive analyses were conducted on three different databases acquired in natural growing conditions: red apples (nine images with 113 apples), green grape clusters (129 images with 1078 grape clusters), and yellow peppers (30 images with 73 peppers). These databases are provided as part of this paper for future developments. The algorithm was evaluated using cross-validation with 70% images for training and 30% images for testing. The algorithm successfully detected apples and peppers in variable lighting conditions resulting with an F-score of 93.17% and 99.31% respectively. Results show the importance of the tuning process for the generalization of the algorithm to different kinds of fruits and environments. In addition, this research revealed the importance of evaluating different color spaces since for each kind of fruit, a different color space might be superior over the others. The LAB color space is most robust to noise. The algorithm is robust to changes in the threshold learned by the training process and to noise effects in images.

show abstract

Section: Introductionsupporting

confidence: 79%

Section: Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Parameter Tuning for Adaptive Thresholding in Fruit Detection

Zemmour

Kurtser

Edan

2019

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Currently, some work has been done to detect the peduncle based on the color information using RGB cameras [6,7,29,30]; however, such cameras are not capable of discriminating between peduncles, leaves, and crops if they are in the same color [7]. The work in [31] proposed a dynamic thresholding to detect apples in viable lighting conditions, and they further used a dynamic adaptive thresholding algorithm [32] for fruit detection using a small set of training images. The work in [27] facilitates detection of peduncles of sweet peppers using multi-spectral imagery; however, the accuracy is too low to be of practical use.…”

Section: Perceptionmentioning

confidence: 99%

An Autonomous Fruit and Vegetable Harvester with a Low-Cost Gripper Using a 3D Sensor

Zhang

Huang

You

et al. 2019

Sensors

View full text Add to dashboard Cite

Reliable and robust systems to detect and harvest fruits and vegetables in unstructured environments are crucial for harvesting robots. In this paper, we propose an autonomous system that harvests most types of crops with peduncles. A geometric approach is first applied to obtain the cutting points of the peduncle based on the fruit bounding box, for which we have adapted the model of the state-of-the-art object detector named Mask Region-based Convolutional Neural Network (Mask R-CNN). We designed a novel gripper that simultaneously clamps and cuts the peduncles of crops without contacting the flesh. We have conducted experiments with a robotic manipulator to evaluate the effectiveness of the proposed harvesting system in being able to efficiently harvest most crops in real laboratory environments. Sensors 2020, 20, 93 2 of 15 the crop peduncle based on the Mask Region-based Convolutional Neural Network (Mask R-CNN) algorithm [10].The rest of this paper is organized as follows: In Section 2, we introduce related work. In Section 3, we describe the design of the gripper and provide a mechanical analysis. In Section 4, we present the approach of cutting-point detection. In Section 5, we describe an experimental robotic system to demonstrate the harvesting process for artificial and real fruits and vegetables. Conclusions and future work are given in Section 6. Related WorkDuring the last few decades, many systems have been developed for the autonomous harvesting of soft crops, ranging from cucumber [11] and tomato-harvesting robots [12] to sweet-pepper [2,13] and strawberry-picking apparatus [14,15]. The work in [16] demonstrated a sweet-pepper-harvesting robot that achieved a success rate of 6% in unstructured environments. This work highlighted the difficulty and complexity of the harvesting problem. The key research challenges include manipulation tools along with the harvesting process, and perception of target fruits and vegetables with the cutting points.

show abstract

“…1 & 5). This can be done either by using external static sensors or eye-in-hand cameras mounted on a robotic manipulator [9] with different algorithms [4,24,31,35,41,43]. Viewpoints analyses in harvesting robotics indicate that only 60% of the fruit can be detected from a single detection direction [22].…”

Section: Introductionmentioning

confidence: 99%

Evaluation of approach strategies for harvesting robots: Case study of sweet pepper harvesting

Ringdahl

Kurtser

Edan

2018

J Intell Robot Syst

Self Cite

View full text Add to dashboard Cite

Robotic harvesters that use visual servoing must choose the best direction from which to approach the fruit to minimize occlusion and avoid obstacles that might interfere with the detection along the approach. This work proposes different approach strategies, compares them in terms of cycle times, and presents a failure analysis methodology of the different approach strategies. The different approach strategies are: in-field assessment by human observers, evaluation based on an overview image using advanced algorithms or remote human observers, or attempting multiple approach directions until the fruit is successfully reached. In the latter approach, each attempt costs time, which is a major bottleneck in bringing harvesting robots into the market. Alternatively, a single approach strategy that only attempts one direction can be applied if the best approach direction is known a-priori. The different approach strategies were evaluated for a case study of sweet pepper harvesting in laboratorial and greenhouse conditions. The first experiment, conducted in a commercial greenhouse, revealed that the fruit approach cycle time increased 8% and 116% for reachable and unreachable fruits respectively when the multiple approach strategy was applied, compared to the single approach strategy. The second experiment measured human observers' ability to provide insights to approach directions based on overview images taken in both greenhouse and laboratorial conditions. Results revealed that human observers are accurate in detecting unapproachable directions while they tend to miss approachable directions. By detecting fruits that are unreachable (via automatic algorithms or human operators), harvesting cycle times can be significantly shortened leading to improved commercial feasibility of harvesting robots.

show abstract

Dynamic thresholding algorithm for robotic apple detection

Cited by 6 publications

References 9 publications

Automatic Parameter Tuning for Adaptive Thresholding in Fruit Detection

Automatic Parameter Tuning for Adaptive Thresholding in Fruit Detection

An Autonomous Fruit and Vegetable Harvester with a Low-Cost Gripper Using a 3D Sensor

Evaluation of approach strategies for harvesting robots: Case study of sweet pepper harvesting

Contact Info

Product

Resources

About