AdaptiveON: Adaptive Outdoor Local Navigation Method For Stable and Reliable Actions

Abstract

We present a novel outdoor navigation algorithm to generate stable and efficient actions to navigate a robot to the goal. We use a multi-stage training pipeline and show that our model produces policies that result in stable and reliable robot navigation on complex terrains. Based on the Proximal Policy Optimization (PPO) algorithm, we developed a novel method to achieve multiple capabilities for outdoor navigation tasks, namely: alleviating the robot's drifting, keeping the robot stable on bumpy terrains, avoiding climbing on hills with steep elevation changes, and collision avoidance. Our training process mitigates the reality(sim-to-real) gap by introducing more generalized environmental and robotic parameters and training with rich features of Lidar perception in the Unity simulator. We evaluate our method in both simulation and the real world with Clearpath Husky and Jackal. Additionally, we compare our method against the state-of-the-art approaches and show that in the real world it improves stability by at least 30.7% on uneven terrains, reduces drifting by 8.08%, and for high hills our trained policy keeps small changes of the elevation of the robot at each motion step by preventing the robot from moving on areas with high gradients.

GP-Frontier for Local Mapless Navigation

“…Two sets of performance metrics are considered for uneven terrain navigation: safety-related metrics and motion-related metrics. The safety-related metrics encompass the robot's roll ϕ and pitch ψ angles, the change in elevation żr during the robot's movement from x s to x f , and the vibration v b , as defined in [29] as the cumulative rate of change for the roll and pitch angles: v b = |ω r |+|ω p |, with ω r = φ and ω p = ψ. A lower vibration value is indicative of a more stable trajectory.…”

HTRON:Efficient Outdoor Navigation with Sparse Rewards via Heavy Tailed Adaptive Reinforce Algorithm

“…Learning-based approaches, in particular DRL have pushed the boundaries of mobile robot navigation for unstructured and dynamically changing environments [25,26,8]. Nevertheless, these methods suffer from sample efficiency under sparse rewards settings because dense rewards play an important role in learning for DRL methods.…”

“…For example, deterministic policies such as DDPG [26], A3C [35], and DQN [36] with dense rewards have been incorporated for robot navigation on uneven terrains. An adaptive model that directly learns control and environmental dynamics is presented in [8]. Moreover, segmentation [37] and self supervised [25] methods for identifying navigable regions have also been utilized with a combination of DRL based navigation methods to ensure stable and efficient navigation.…”

“…The majority of the RL results revolve around learning in standard environments and platforms like Gym, Mujoco [44] or other simulated environments where the rewards structures are already defined. However, in practical outdoor navigation scenarios, it is extremely hard to generate dense rewards due to the possibility of huge unknown trajectories a robot can take while navigating in the environment [34,8]. The problem is exacerbated in continuous state-action spaces where one needs to define rewards even for infinite possibilities, which could be impractical.…”

“…Such applications require robots to gather useful information efficiently from the environment to make intelligent navigation decisions [4]. To this end, Deep Reinforcement Learning (DRL) techniques have been widely employed in recent robotic systems due to their inherent exploration and exploitation capabilities to gather necessary information from the learning environments [5,6,7,8,9,10]. Nevertheless, a major challenge for DRL based algorithms is dealing with sparse rewards in continuous state and actions spaces.…”