Lidar-Based 3D Obstacle Detection Using Focal Voxel R-CNN for Farmland Environment

Abstract

With advances in precision agriculture, autonomous agricultural machines can reduce human labor, optimize workflow, and increase productivity. Accurate and reliable obstacle-detection and avoidance systems are essential for ensuring the safety of automated agricultural machines. Existing LiDAR-based obstacle detection methods for the farmland environment process the point clouds via manually designed features, which is time-consuming, labor-intensive, and weak in terms of generalization. In contrast, deep learning has a powerful ability to learn features autonomously. In this study, we attempted to apply deep learning in LiDAR-based 3D obstacle detection for the farmland environment. In terms of perception hardware, we established a data acquisition platform including LiDAR, a camera, and a GNSS/INS on the agricultural machine. In terms of perception method, considering the different agricultural conditions, we used our datasets to train an effective 3D obstacle detector, known as Focal Voxel R-CNN. We used focal sparse convolution to replace the original 3D sparse convolution because of its adaptable ability to extract effective features from sparse point cloud data. Specifically, a branch of submanifold sparse convolution was added to the upstream of the backbone convolution network; this adds weight to the foreground point and retains more valuable information. In comparison with Voxel R-CNN, the proposed Focal Voxel R-CNN significantly improves the detection performance for small objects, and the AP in the pedestrian class increased from 89.04% to 92.89%. The results show that our model obtains an mAP of 91.43%, which is 3.36% higher than the base model. The detection speed is 28.57 FPS, which is 4.18 FPS faster than the base model. The experiments show the effectiveness of our model, which can provide a more reliable obstacle detection model for autonomous agricultural machines.

New Trends and Challenges in Precision and Digital Agriculture

“…In order to obtain more precise and accurate results, efforts are being made to improve existing machine learning methods. The ResNet50 model, Faster RCNN, and Focal Voxel R-CNN have been used to diagnose rice blight, detect defects in groundnut crops, and detect obstacles by automated agricultural machinery, respectively [42][43][44]. Diagnosing agricultural crop diseases or detecting obstacles in real time is not possible without advanced visual technologies.…”

Hierarchical Detection of Gastrodia elata Based on Improved YOLOX

“…Currently, with the development of computer vision, deep learning has been widely applied in agriculture [4][5][6], medicine [7][8][9], and other fields. Though the classification of Gastrodia elata takes both weights and shape into consideration, it is only sorted by manual experience or only considering weight, leading to low sorting accuracy and a heavy workload.…”

Three-dimensional reconstruction of the furrow shape in orchards using a low-cost lidar

“…Lidar overcomes the shortcoming of traditional sensors in obtaining spatial information and can work all day long to obtain the point cloud data of real trench spatial information. Lidar has been widely used in obstacle detection (Qin et al, 2023;Shang et al, 2023), terrain mapping (Kim and Choi, 2021;García-López et al, 2023), map construction (Su et al, 2021;Ao et al, 2022;Eisoldt et al, 2022;Rivera et al, 2023), agricultural information monitoring, and plant model reconstruction (Perez et al, 2018;Tsoulias et al, 2019;Campbell et al, 2023).…”