Deep-Learning-Based Trunk Perception with Depth Estimation and DWA for Robust Navigation of Robotics in Orchards

Abstract

Agricultural robotics is a complex, challenging, and exciting research topic nowadays. However, orchard environments present harsh conditions for robotics operability, such as terrain irregularities, illumination, and inaccuracies in GPS signals. To overcome these challenges, reliable landmarks must be extracted from the environment. This study addresses the challenge of accurate, low-cost, and efficient landmark identification in orchards to enable robot row-following. First, deep learning, integrated with depth information, is used for real-time trunk detection and location. The in-house dataset used to train the models includes a total of 2453 manually annotated trunks. The results show that the trunk detection achieves an overall mAP of 81.6%, an inference time of 60 ms, and a location accuracy error of 9 mm at 2.8 m. Secondly, the environmental features obtained in the first step are fed into the DWA. The DWA performs reactive obstacle avoidance while attempting to reach the row-end destination. The final solution considers the limitations of the robot’s kinematics and dynamics, enabling it to maintain the row path and avoid obstacles. Simulations and field tests demonstrated that even with a certain initial deviation, the robot could automatically adjust its position and drive through the rows in the real orchard.

Intrarow Uncut Weed Detection Using You-Only-Look-Once Instance Segmentation for Orchard Plantations

“…However, most of the vision systems utilized were not trained, which may lead to failure when illumination is insufficient. Therefore, intelligent vision systems such as deep learning [21], which are pretrained even in the presence of limited illumination, demonstrate satisfactory accuracy.…”

Advancements in Learning-Based Navigation Systems for Robotic Applications in MRO Hangar: Review

“…This limitation can result in challenges when attempting to accurately track fast-moving objects in real time. RGBD cameras have also shown great capabilities, including high resolution and generation of rich and detailed environment information, though within a limited range, but are greatly efficient in object position estimation using depth information [ 60 , 61 ]. However, the performance is highly susceptible to lightning conditions, which can be associated with certain areas in the hangar environments.…”

New Trends and Challenges in Precision and Digital Agriculture

“…Precision fertilization, irrigation, current assessments of vegetation condition, decision support and management systems, yield prediction, and the classification of disease signs and symptoms are implemented in greenhouse crops, production fields, orchards, vegetation halls, etc. [4,37,[54][55][56][57][58]. The above solutions are practiced extensively in cereal and rapeseed crops, cotton, and soybeans [15,59,60].…”