RumexWeeds: A grassland dataset for agricultural robotics

Abstract

Computer vision can lead toward more sustainable agricultural production by enabling robotic precision agriculture. Vision-equipped robots are being deployed in the fields to take care of crops and control weeds. However, publicly available agricultural datasets containing both image data as well as data from navigational robot sensors are scarce. Our real-world dataset RumexWeeds targets the detection of the grassland weeds: Rumex obtusifolius L. and Rumex crispus L. RumexWeeds includes whole image sequences instead of individual static images, which is rare for computer vision image datasets, yet crucial for robotic applications. It allows for more robust object detection, incorporating temporal aspects and considering different viewpoints of the same object. Furthermore, RumexWeeds includes data from additional navigational robot sensors-GNSS, IMU and odometry-which can increase robustness, when additionally fed to detection models. In total the dataset includes 5510 images with 15,519 manual bounding box annotations collected at three different farms and four different days in summer and autumn 2021.Additionally, RumexWeeds includes a subset of 340 ground truth pixels-wise annotations. The dataset is publicly available at https://dtu-pas.github.io/ RumexWeeds/. In this paper we also use RumexWeeds to provide baseline weed detection results considering a state-of-the-art object detector; in this way we are elucidating interesting characteristics of the dataset.

Real-Time Joint-Stem Prediction for Agricultural Robots in Grasslands Using Multi-Task Learning

“…Weed detection has been widely investigated in the past decade and it has been posed as a classification problem [1][2][3][4][5][6], detection problem via bounding boxes [7][8][9][10][11], as well as segmentation problem [12][13][14][15][16]. The object detection architecture YOLO has evolved greatly over the last decade resulting in different popular variants, such as YOLOv5 [17], YOLOX [18], and YOLOv8 [19], which have been widely applied to the agricultural domain [11,[20][21][22][23].…”

“…The publicly available grassland weed dataset RumexWeeds [11] is considered in this work. It is a real-world dataset, targeting the most problematic grassland weed, Rumex.…”

Few-leaf Learning: Weed Segmentation in Grasslands

“…For the data collection, we used a Clearpath Husky robot as shown in Figure 4, which is specified in more detail in [7]. The robot is equipped with two vision sensors: a forward-looking RGB camera in order to roughly detect the Rumex weeds [7] and a close-up IntelRealsense L515 LiDAR. Once a weed plant is detected on the RGB camera imagery, the robot drives over the plant to perform a fine-grained analysis of the plant and treat it accordingly.…”

“…Moreover, each datapoint is accompanied by navigational information such as Odometry, GNSS, and IMU of the robot. For the data collection, we used a Clearpath Husky robot as shown in Figure 4, which is specified in more detail in [7]. The robot is equipped with two vision sensors: a forward-looking RGB camera in order to roughly detect the Rumex weeds [7] and a close-up IntelRealsense L515 LiDAR.…”

“…On the contrary, when the keypoints deviate of same distance but along the line, POKS evaluates it with higher more reasonable scores. We showcase our work in the domain of grassland, where we extract plant traits for a highly problematic weed Rumex obtusifolius L. [6], [7]. Compared to crop fields, grasslands constitute additional challenges with more cluttered background, thin grass halms that can easily be confused with the stems as well as more occlusions from the background.…”