Robust visual servo control in the presence of fruit motion for robotic citrus harvesting

Mehta, Siddhartha S.; MacKunis, William; Burks, Thomas F.

doi:10.1016/j.compag.2016.03.007

Cited by 63 publications

(53 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A support vector machine performed the actual classification of the broccoli heads (Kusumam et al, 2016). The use of vision to provide control through methods including visual servoing has also been shown to increase positional accuracy when harvesting citrus fruit (Mehta & Burks, 2014;Mehta, MacKunis, & Burks, 2016).…”

mentioning

confidence: 99%

A field‐tested robotic harvesting system for iceberg lettuce

Birrell

Hughes

Cai

et al. 2019

Journal of Field Robotics

131

View full text Add to dashboard Cite

Agriculture provides an unique opportunity for the development of robotic systems; robots must be developed which can operate in harsh conditions and in highly uncertain and unknown environments. One particular challenge is performing manipulation for autonomous robotic harvesting. This paper describes recent and current work to automate the harvesting of iceberg lettuce. Unlike many other produce, iceberg is challenging to harvest as the crop is easily damaged by handling and is very hard to detect visually. A platform called Vegebot has been developed to enable the iterative development and field testing of the solution, which comprises of a vision system, custom end effector and software. To address the harvesting challenges posed by iceberg lettuce a bespoke vision and learning system has been developed which uses two integrated convolutional neural networks to achieve classification and localization. A custom end effector has been developed to allow damage free harvesting. To allow this end effector to achieve repeatable and consistent harvesting, a control method using force feedback allows detection of the ground. The system has been tested in the field, with experimental evidence gained which demonstrates the success of the vision system to localize and classify the lettuce, and the full integrated system to harvest lettuce. This study demonstrates how existing state‐of‐the art vision approaches can be applied to agricultural robotics, and mechanical systems can be developed which leverage the environmental constraints imposed in such environments.

show abstract

mentioning

confidence: 99%

A field‐tested robotic harvesting system for iceberg lettuce

Birrell

Hughes

Cai

et al. 2019

Journal of Field Robotics

131

View full text Add to dashboard Cite

show abstract

“…A new trend for agricultural robotics, following the recent advances in robotic technology around the world; is to combine and integrate advanced control theory, computer vision algorithms, and machine learning techniques into visual servoing approaches, allowing robots to perform agricultural tasks with a high level of autonomy and better response to decision-making assignments into complex and very dynamic scenarios [3,4,5]. Over the last years, the technological advances of sensors and communication systems have encouraged the agricultural industry to employ and design intelligent autonomous robots to carry out a number of repetitive and dull tasks for farmers in orchards, vineyards, poly-tunnels and farms [7,8].…”

Section: Motivationmentioning

confidence: 99%

“…resulting in the following linear error system: e p + Λ p e p = 0 , (2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17) provided that the kinematic parameter b i ∈ R for i = 1, 2, · · · , n k is assumed to be fully known. Notice that, if Λ p is a positive definite matrix the error system is asymptotically stable and the convergence rate depends on the eigenvalues of Λ p , that is, the larger the eigenvalues, the faster the convergence.…”

Section: Jacobian (Pseudo-)inversementioning

confidence: 99%

“…Moreover, the term J † p (q) is introduced to compensate for J p (q) and linearize the error system, while the termṗ d (t) ensures the convergence of the position error e p to zero, independently of the chosen reference trajectory p d (t) [37]. The feedback control design (2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15) can be easily implemented as illustrated in the block diagram of the inverse kinematics-based algorithm with pseudo-inverse Jacobian (Fig. 2.1).…”

Section: Jacobian (Pseudo-)inversementioning

confidence: 99%

“…A computationally simpler approach consists of finding a suitable relationship betweenq and e p in order to ensure the convergence of the position error to zero without resorting to the system linearization as discussed in (2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15).…”

Section: Jacobian Transposementioning

confidence: 99%

See 2 more Smart Citations

A Robust Visual Servoing Approach for Robotic Fruit Harvesting

CAMACHO¹

View full text Add to dashboard Cite

In this work, we present different eye-in-hand visual servoing control schemes applied to a robotic harvesting task of soft fruits in the presence of parametric uncertainties in the system models. The first scheme combines position-based visual servoing (PBVS) and image-based visual servoing (IBVS) approaches in order to perform respectively an approach phase to the fruit and then a fine tuning of the end-effector to harvest. The second scheme uses a hybrid visual servoing (HVS) approach to fulfill the complete harvesting task, by designing a suitable control law which combines error vectors defined in both the image and operational spaces. For detecting the fruits, an algorithm based on the combination of the OHTA color space and Otsu's threshold method for a fast recognition of mature fruits in complex scenarios. In addition, a more accurate detection method employs a pre-trained deep encoder-decoder algorithm based on a minimized Segnet version for a fast and cheap inference during the task execution. The object localization is accomplished by employing an image triangulation technique, which combines the speeded-up-robust-features (SURF) and the-randomsample-consensus (RANSAC) or the Oriented FAST and Rotated BRIEF and the Brute-Force Matcher (BF-Matcher) algorithms to extract the fruit image feature and match it to its correspondent feature-point into the other view of the stereo camera. However, since these algorithms are computationally expensive for the task requirements, a faster estimation method uses the fruit centroid and a homogeneous transformation for discovering matching points. Finally, a vision-based sliding-mode-control scheme and a switching monitoring function are employed to cope with uncertainties in the calibration parameters of the camera-robot system. In this context, it is possible to guarantee the asymptotic stability and convergence of the image feature error, even if the misalignment angle, around the z-axis, between the camera and end-effector frames is uncertain. 3D computer simulations and preliminary experimental results, obtained with a Mitsubishi robot arm RV-2AJ carrying out a simple strawberry picking task, are included to illustrate the performance and effectiveness of the proposed control scheme.

show abstract

Design and evaluation of a modular robotic plum harvesting system utilizing soft components

Brown

Sukkarieh

2020

Journal of Field Robotics

View full text Add to dashboard Cite

The human labor required for tree crop harvesting is a major cost component in fruit production and is increasing. To address this, many existing research works have sought to demonstrate commercially viable robotic harvesting for tree crops, though successful commercial products resulting from these have been few and far between. Systems developed for specific crops such as sweet peppers or apples have shown promise, but the vast majority of cultivar types remain unaddressed, and developing a specific system for each one is inefficient. In this study, an easily modifiable development platform for robotic fruit harvesting is presented, this can be used to test specific design choices on different fruit and growing conditions. The system is evaluated in a commercial plum orchard, with no crop modifications. Both a hard and soft gripper are trialed, along with three object detector approaches and two picking motions. Some existing techniques are found to be counterproductive for plums, while soft robotics and persistent target tracking significantly improve performance. The best harvest success rate of 42%, was observed when using the soft gripper with complex motion. This is lower than expected based on prior testing with apples and indicates the difficulty in moving to new fruit types. Unique challenges specific to the plum type and growing style are examined in the context of system module design choices.

show abstract

Robust visual servo control in the presence of fruit motion for robotic citrus harvesting

Cited by 63 publications

References 30 publications

A field‐tested robotic harvesting system for iceberg lettuce

A field‐tested robotic harvesting system for iceberg lettuce

A Robust Visual Servoing Approach for Robotic Fruit Harvesting

Design and evaluation of a modular robotic plum harvesting system utilizing soft components

Contact Info

Product

Resources

About