Control of Rough Terrain Vehicles Using Deep Reinforcement Learning

Abstract

We explore the potential to control terrain vehicles using deep reinforcement in scenarios where human operators and traditional control methods are inadequate. This letter presents a controller that perceives, plans, and successfully controls a 16-tonne forestry vehicle with two frame articulation joints, six wheels, and their actively articulated suspensions to traverse rough terrain. The carefully shaped reward signal promotes safe, environmental, and efficient driving, which leads to the emergence of unprecedented driving skills. We test learned skills in a virtual environment, including terrains reconstructed from high-density laser scans of forest sites. The controller displays the ability to handle obstructing obstacles, slopes up to 27 • , and a variety of natural terrains, all with limited wheel slip, smooth, and upright traversal with intelligent use of the active suspensions. The results confirm that deep reinforcement learning has the potential to enhance control of vehicles with complex dynamics and high-dimensional observation data compared to human operators or traditional control methods, especially in rough terrain.

A Comprehensive Overview of Control Algorithms, Sensors, Actuators, and Communication Tools of Autonomous All-Terrain Vehicles in Agriculture

“…With Kahn's algorithm, robot navigation policies can be learned in a self-supervised manner that requires minimal human interaction, using samples in an efficient, stable, and highperforming manner [120]. Deep RL is capable of learning control for rough terrain vehicles that have continuous, high-dimensional observations and actions in their environment, according to Wiberg [121].…”

Analytic Solutions for Wheeled Mobile Manipulator Supporting Forces

“…When a wheeled platform is transiting an uneven terrain, strictly speaking, in the general case, each wheel is experiencing different ground slope values and gradients. This traversing introduces an error in the calculations if the local terrain slopes beneath each wheel are not obtained and are not accounted for correctly, as could be done using reconstruction from high-density laser scans [40].…”

Analytic Solutions for Wheeled Mobile Manipulator Supporting Forces

“…When a wheeled platform is transiting an uneven terrain, strictly speaking, in the general case, each wheel is experiencing different ground slope values and gradients. This traversing introduces error in calculations if the local terrain slopes beneath each wheel are not obtained and accounted for correctly, as could be done using reconstruction from high-density laser scans [33]. Not accounting for the terrain properties in detail may not be that significant, especially in the case of gentler terrains with no ruggedness.…”