Perception-aware Path Planning for UAVs using Semantic Segmentation

Abstract

In this work, we present a perception-aware pathplanning pipeline for Unmanned Aerial Vehicles (UAVs) for navigation in challenging environments. The objective is to reach a given destination safely and accurately by relying on monocular camera-based state estimators, such as Keyframebased Visual-Inertial Odometry (VIO) systems. Motivated by the recent advances in semantic segmentation using deep learning, our path-planning architecture takes into consideration the semantic classes of parts of the scene that are perceptually more informative than others. This work proposes a planning strategy capable of avoiding both texture-less regions and problematic areas, such as lakes and oceans, that may cause large drift or failures in the robot's pose estimation, by using the semantic information to compute the next best action with respect to perception quality. We design a hierarchical planner, composed of an A * path-search step followed by B-Spline trajectory optimization. While the A * steers the UAV towards informative areas, the optimizer keeps the most promising landmarks in the camera's field of view. We extensively evaluate our approach in a set of photo-realistic simulations, showing a remarkable improvement with respect to the state-of-the-art in active perception.

Informed Sampling Exploration Path Planner for 3D Reconstruction of Large Scenes

“…In contrast to RRT methods, our set of evaluated viewpoints is unconnected, meaning that we employ a separate path-planner to guide the MAV to the NBV. To this end, we adapt the perception-aware path-planner proposed in [13] to guide the MAV to its goal, while simultaneously directing the sensor at nearby frontiers.…”

“…The perception-aware path-planner presented in [13] uses a two-stage planning process with an initial A * path search based on motion primitives followed by a B-Spline trajectory optimization. Initially, A * searches a collision free path only considering the MAV's position in R 3 .…”

“…The orientation is considered in the following B-Spline optimization, which optimizes the path for collision avoidance, smoothness, and visibility of frontiers. We extend the state space from [x, y, z, γ] to [x, y, z, γ, θ] while keeping the same cost formulations as in [13], thereby including yaw and pitch in the gradient based optimization of the trajectory. The frontier visibility cost favours orientations with more visible frontier voxels.…”

Semantic-aware Active Perception for UAVs using Deep Reinforcement Learning

“…One of the most relevant rival to this work is the perception-aware planner in [8], proposing to generate motion primitives and evaluate them by considering the concentration of landmarks in each area, the probability of collision and the distance to the goal. In [11], however, it is demonstrated that landmark concentration alone is not enough to identify the best areas to fly through, and instead, a perception-aware planner employing semantics to evaluate the quality of the candidate areas for navigation is proposed. Using the semantic segmentation of the SLAM input image as an additional cue in a perception-aware planning algorithm, [11] was shown to achieve the best results in terms of accuracy and robustness of localization to date.…”

“…In [11], however, it is demonstrated that landmark concentration alone is not enough to identify the best areas to fly through, and instead, a perception-aware planner employing semantics to evaluate the quality of the candidate areas for navigation is proposed. Using the semantic segmentation of the SLAM input image as an additional cue in a perception-aware planning algorithm, [11] was shown to achieve the best results in terms of accuracy and robustness of localization to date. Nonetheless, it is not able to adapt dynamically to changes in the navigation area at flight time, as it assigns fixed binary informativeness scores to every semantic class in the scene.…”