Semantic-aware Active Perception for UAVs using Deep Reinforcement Learning

Abstract

This work presents a semantic-aware pathplanning pipeline for Unmanned Aerial Vehicles (UAVs) using deep reinforcement learning for vision-based navigation in challenging environments. Driven by the maturity of works in semantic segmentation, the proposed path-planning architecture uses reinforcement learning to distinguish the parts of the scene that are perceptually more informative using semantic cues, in effect guiding more robust, repeatable, and accurate navigation of the UAV to the predefined goal destination. Assuming that the UAV performs vision-based state estimation, such as keyframebased visual odometry, and semantic segmentation onboard, the proposed deep policy network continuously evaluates the optimal relative perceptual informativeness of each semantic class in view. A perception-aware path planner uses these informativeness values to perform trajectory optimization in order to generate the next best action with respect to the current state and the perception quality of the surroundings, essentially guiding the UAV to avoid flying over perceptually degraded regions. Thanks to the use of semantic cues, the policy can be trained in a large number of non-photorealistic randomly-generated scenes, and results to an architecture that is generalizable to environments with the same semantic classes, independently of their visual appearance. Extensive evaluations on challenging, photorealistic simulations reveal a remarkable improvement in robustness and success rate with the proposed approach over the state of the art in active perception.Videohttps://youtu.be/RaO3whUBVnc

Autonomous Emergency Landing for Multicopters using Deep Reinforcement Learning

“…obstacles, free space), semantics expose the agent to the spatial relationships between different classes (e.g. cars and roads; houses and grass) [6], speeding up the search for valid landing spots. Moreover, using these mid-level representations as input to a deep RL agent is proven to yield better generalization of the policy [19].…”

“…This is performed by leveraging the most recent results in deep learning for depth completion [17] and semantic segmentation [18]. Using these mid-level representations as inputs can alleviate the simulation-to-reality gap, allowing for faster and more stable training of deep RL agents [6], [19]. We demonstrate that our policy, trained exclusively in simulation, can be directly deployed onboard a UAV in real-world missions.…”

“…To further analyze our pipeline, we run ablation studies on the policy architecture and type of input to the agent. We compare our approach against two variants; the Ours + Memory policy with an additional Long Short-Term Memory (LSTM) component with 256 cells before the Actor-Critic module (similarly to [6]), and a Binary policy with the same architecture as in Fig. 3, but trained with depth images and binary semantic masks, where the only valid landing area is 'terrain'.…”

“…Extending the policy with the memory module exhibits the worst performances at both training and testing time, with no successful runs in 3 out of 7 test models. While LSTMs could be beneficial in dynamic scenes [6], they require more time to train, leading to lower rewards. Instead, the Binary policy converges faster, but to a lower average reward than our policy.…”

“…Moreover, these methods reason only about the geometry of the navigation area dismissing more general and relevant context, for example, the fact that some areas can be identified as clear spaces, but are unsuitable for landing, such as rooftops and roads. In order to overcome this issue, pipelines use high-level semantic information, which has been receiving growing attention in recent years [5], as semantic segmentation has been proven to be extremely useful for path-planning applications [6]. However, these approaches have been deployed in simulation only, raising questions about their applicability in real-world missions.…”