PATHoBot: A Robot for Glasshouse Crop Phenotyping and Intervention

Smitt, Claus; Halstead, Michael; Zaenker, Tobias; Bennewitz, Maren; McCool, Chris

doi:10.1109/icra48506.2021.9562047

Cited by 32 publications

(26 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Horticulture Glasshouse Sweet Pepper (BUP20) consists of video sequences from a glasshouse environment in CKA, with two sweet pepper cultivar Mavera (yellow) and Allrounder (red), each cultivar matured from green, to mixed, to their primary color. This data, which contains all the colors, was captured by the autonomous phenotyping platform PATHoBot [2] driving at 0.2m/s. The dataset comprises 10 sequences of 6 crop-rows, captured using Intel RealSense D435i cameras recording RGB-D images, IMU data and wheel odometry.…”

Section: A Datasetsmentioning

confidence: 99%

“…In recent years, the adoption of robotic systems in agriculture has seen a significant surge. This uptake has been driven by recent state-of-the-art research to automate tasks such as weeding [1], yield estimation [2] and harvesting [3], [4]. Key to these advances has been the novel vision systems to enable scene understanding and crop monitoring.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Explicitly Incorporating Spatial Information to Recurrent Networks for Agriculture

Smitt

Halstead

Ahmadi

et al. 2022

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

In agriculture, the majority of vision systems perform still image classification. Yet, recent work has highlighted the potential of spatial and temporal cues as a rich source of information to improve the classification performance. In this paper, we propose novel approaches to explicitly capture both spatial and temporal information to improve the classification of deep convolutional neural networks. We leverage available RGB-D images and robot odometry to perform inter-frame feature map spatial registration. This information is then fused within recurrent deep learnt models, to improve their accuracy and robustness. We demonstrate that this can considerably improve the classification performance with our best performing spatial-temporal model (ST-Atte) achieving absolute performance improvements for intersection-over-union (IoU[%]) of 4.7 for crop-weed segmentation and 2.6 for fruit (sweet pepper) segmentation. Furthermore, we show that these approaches are robust to variable framerates and odometry errors, which are frequently observed in real-world applications.

show abstract

Section: A Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Explicitly Incorporating Spatial Information to Recurrent Networks for Agriculture

Smitt

Halstead

Ahmadi

et al. 2022

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

show abstract

“…MOT methods have been recently used in agro-food robotic systems (Halstead et al, 2018;Kirk et al, 2021;Smitt et al, 2021;Halstead et al, 2021). Nevertheless, their objectives were to perform specific tasks like plant phenotyping and monitoring, and were not aiming for a generic representation to improve the capabilities of an autonomous robotic system in tasks like harvesting and crop maintenance.…”

Section: Introductionmentioning

confidence: 99%

“…Their goal was fruit counting and ripeness estimation. The work of Halstead et al (2018) was extended by Smitt et al (2021) to autonomously count fruits over a greenhouse plant row. They made use of the 3D data obtained from the camera mounted in the robotic system to re-project the bounding boxes of tracked objects onto the following frames, then the tracking was performed by the previously mentioned IoU-based system.…”

Section: Introductionmentioning

confidence: 99%

“…They made use of the 3D data obtained from the camera mounted in the robotic system to re-project the bounding boxes of tracked objects onto the following frames, then the tracking was performed by the previously mentioned IoU-based system. Halstead et al (2021) further extended the work of Smitt et al (2021) to different cropping systems, like arable farmland, and evaluated the performance on sugar beet plant counting. Kirk et al (2021) also developed an IoU-based tracking system assisted by deep learning, inspired by the work done by Wojke et al (2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Development and evaluation of automated localization and reconstruction of all fruits on tomato plants in a greenhouse based on multi-view perception and 3D multi-object tracking

Rincon¹,

Henten²,

Kootstra³

2022

Preprint

View full text Add to dashboard Cite

Accurate representation and localization of relevant objects is important for robots to perform tasks. Building a generic representation that can be used across different environments and tasks is not easy, as the relevant objects vary depending on the environment and the task. Furthermore, another challenge arises in agro-food environments due to their complexity, and high levels of clutter and occlusions. In this paper, we present a method to build generic representations in highly occluded agro-food environments using multi-view perception and 3D multi-object tracking. Our representation is built upon a detection algorithm that generates a partial point cloud for each detected object. The detected objects are then passed to a 3D multi-object tracking algorithm that creates and updates the representation over time. The whole process is performed at a rate of 10 Hz. We evaluated the accuracy of the representation on a real-world agro-food environment, where it was able to successfully represent and locate tomatoes in tomato plants despite a high level of occlusion. We were able to estimate the total count of tomatoes with a maximum error of 5.08% and to track tomatoes with a tracking accuracy up to 71.47%. Additionally, we showed that an evaluation using tracking metrics gives more insight in the errors in localizing and representing the fruits.

show abstract

Robotic Vegetable Production

2023

Encyclopedia of Smart Agriculture Technologies

View full text Add to dashboard Cite

PATHoBot: A Robot for Glasshouse Crop Phenotyping and Intervention

Cited by 32 publications

References 18 publications

Explicitly Incorporating Spatial Information to Recurrent Networks for Agriculture

Explicitly Incorporating Spatial Information to Recurrent Networks for Agriculture

Development and evaluation of automated localization and reconstruction of all fruits on tomato plants in a greenhouse based on multi-view perception and 3D multi-object tracking

Robotic Vegetable Production

Contact Info

Product

Resources

About