Viewpoint Planning for Fruit Size and Position Estimation

Zaenker, Tobias; Smitt, Claus; McCool, Chris; Bennewitz, Maren

doi:10.1109/iros51168.2021.9636701

Cited by 29 publications

(24 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For our evaluation in simulated scenarios, we used the Gazebo based simulation setup presented in [8], but with an additional new environment with more and different plants. It simulates a UR5e arm from Universal Robots with an RGB-D camera placed at its flange.…”

Section: Methodsmentioning

confidence: 99%

“…As an example, Roy and Isler [12] used a viewpoint planning approach to count apples in orchards, however, without any volumetric estimation. In our previous work [8], we formulated a sampling approach based on regions of interest (ROI), i.e., fruits that outperforms those that do not consider ROIs, with respect to the number of detected fruits and the estimated volume. However, the previous approach suffered from inaccuracies in fruit volume estimation as no shape estimation or modeling was carried out for predicting the unobserved fruit regions.…”

Section: Related Workmentioning

confidence: 99%

“…KinFu can be used for realtime mapping of scenes while concurrently obtaining the camera pose by tracking the depth frames continuously as in [7]. However, as the viewpoint planner used in our previous approach [8] plans discrete viewpoints for maximising information gain, KinFu cannot perform continuous tracking and registration which needs small movements in successive frames for the convergence of ICP alignment [20]. Oleynikova et al [9] developed voxblox to build TSDFs based map representations for local planning of Micro Aerial Vehicles.…”

Section: Related Workmentioning

confidence: 99%

“…Our approach aims at estimating the shape of detected fruits, which might be partially occluded. We use our previously developed viewpoint planner [8] to move a camera placed on a robotic arm, which enables us to observe fruits from different perspectives. In order to estimate the shape of detected fruits, we build a map of the environment.…”

Section: System Overviewmentioning

confidence: 99%

“…We build upon ideas of Lehnert et al [7] who proposed to match the shape of sweet peppers with superellipsoids. Whereas their experiments were limited to controlled environments with single pepper plants, we integrated our superellipsoid matching system with our previously developed viewpoint planner [8]. As a result, we are able to apply our fruit mapping system on data captured with an RGB-D camera on a robotic arm in various scenarios containing multiple plants with sweet peppers.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Fruit Mapping with Shape Completion for Autonomous Crop Monitoring

Marangoz¹,

Zaenker²,

Menon³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous crop monitoring is a difficult task due to the complex structure of plants. Occlusions from leaves can make it impossible to obtain complete views about all fruits of, e.g., pepper plants. Therefore, accurately estimating the shape and volume of fruits from partial information is crucial to enable further advanced automation tasks such as yield estimation and automated fruit picking. In this paper, we present an approach for mapping fruits on plants and estimating their shape by matching superellipsoids. Our system segments fruits in images and uses their masks to generate point clouds of the fruits. To combine sequences of acquired point clouds, we utilize a real-time 3D mapping framework and build up a fruit map based on truncated signed distance fields. We cluster fruits from this map and use optimized superellipsoids for matching to obtain accurate shape estimates.In our experiments, we show in various simulated scenarios with a robotic arm equipped with an RGB-D camera that our approach can accurately estimate fruit volumes. Additionally, we provide qualitative results of estimated fruit shapes from data recorded in a commercial glasshouse environment.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: System Overviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Fruit Mapping with Shape Completion for Autonomous Crop Monitoring

Marangoz¹,

Zaenker²,

Menon³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

NDT‐6D for color registration in agri‐robotic applications

Gupta,

Lilienthal,

Andreasson

et al. 2023

Journal of Field Robotics

View full text Add to dashboard Cite

Registration of point cloud data containing both depth and color information is critical for a variety of applications, including in-field robotic plant manipulation, crop growth modeling, and autonomous navigation. However, current state-ofthe-art registration methods often fail in challenging agricultural field conditions due to factors such as occlusions, plant density, and variable illumination.To address these issues, we propose the NDT-6D registration method, which is a color-based variation of the Normal Distribution Transform (NDT) registration approach for point clouds. Our method computes correspondences between pointclouds using both geometric and color information and minimizes the distance between these correspondences using only the three-dimensional (3D) geometric dimensions. We evaluate the method using the GRAPES3D data set collected with a commercial-grade RGB-D sensor mounted on a mobile platform in a vineyard. Results show that registration methods that only rely on depth information fail to provide quality registration for the tested data set. The proposed color-based variation outperforms state-of-the-art methods with a root mean square error (RMSE) of 1.1-1.6 cm for NDT-6D compared with 1.1-2.3 cm for other color-information-based methods and 1.2-13.7 cm for noncolor-information-based methods. The proposed method is shown to be robust against noises using the TUM RGBD data set by artificially adding noise present in an outdoor scenario. The relative pose error (RPE) increased ~14% for our method compared to an increase of ~75% for the best-performing registration method. The obtained average accuracy suggests that the NDT-6D registration methods can be used for in-field precision agriculture applications, for example, crop detection, size-based maturity estimation, and growth modeling.

show abstract

Automated Accessibility Map Construction with Two-Level Viewpoint Planning Strategy

Saito,

Miura

2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Viewpoint Planning for Fruit Size and Position Estimation

Cited by 29 publications

References 16 publications

Fruit Mapping with Shape Completion for Autonomous Crop Monitoring

Fruit Mapping with Shape Completion for Autonomous Crop Monitoring

NDT‐6D for color registration in agri‐robotic applications

Automated Accessibility Map Construction with Two-Level Viewpoint Planning Strategy

Contact Info

Product

Resources

About