PAMPC: Perception-Aware Model Predictive Control for Quadrotors

Abstract

We present the first perception-aware model predictive control framework for quadrotors that unifies control and planning with respect to action and perception objectives. Our framework leverages numerical optimization to compute trajectories that satisfy the system dynamics and require control inputs within the limits of the platform. Simultaneously, it optimizes perception objectives for robust and reliable sensing by maximizing the visibility of a point of interest and minimizing its velocity in the image plane. Considering both perception and action objectives for motion planning and control is challenging due to the possible conflicts arising from their respective requirements. For example, for a quadrotor to track a reference trajectory, it needs to rotate to align its thrust with the direction of the desired acceleration. However, the perception objective might require to minimize such rotation to maximize the visibility of a point of interest. A model-based optimization framework, able to consider both perception and action objectives and couple them through the system dynamics, is therefore necessary. Our perception-aware model predictive control framework works in a receding-horizon fashion by iteratively solving a non-linear optimization problem. It is capable of running in real-time, fully onboard our lightweight, small-scale quadrotor using a low-power ARM computer, together with a visual-inertial odometry pipeline. We validate our approach in experiments demonstrating (I) the conflict between perception and action objectives, and (II) improved behavior in extremely challenging lighting conditions. SUPPLEMENTARY MATERIAL Video: https://youtu.be/9vaj829vE18 Code: https://github.com/uzh-rpg/rpg_mpc * These authors contributed equally to this manuscript.

Active Perception Based Formation Control for Multiple Aerial Vehicles

“…However, their method only identifies suboptimal control inputs due to the non-convexity of optimization and the lack of collision avoidance guarantees. In [11], a perception-aware MPC generates real-time motion plans which maximize the visibility of a desired target. The motion plans are, however, generated only for a single aerial robot.…”

Aggressive Perception-Aware Navigation Using Deep Optical Flow Dynamics and PixelMPC

“…In the visual servoing literature, to the best of our knowledge, the real-time predictive controllers used for a visual tracking task are [15], [16]. Although [15] formulated an c 2020 IEEE.…”

“…Thanks to the great success in the field of computer vision, we can use real-time object detectors [17], [18] with GPUs. Although our method requires prior knowledge of the image of the targets (gates) and a trained detector, we believe this is less restrictive than full knowledge of the global 3D position of features like in [16]. Furthermore, we believe our case is less restrictive since our proposed approach can be used for any moving target objects located anywhere in the scene.…”

Joint Vision-Based Navigation, Control and Obstacle Avoidance for UAVs in Dynamic Environments

“…On the other hand, thanks to the progresses in perception and control algorithms [7], [16], and to the increased computational capabilities of embedded computers, vision-based optimal control techniques became a standard for UAVs moving in dynamic environments [2], [24]. They allow to mitigate some of the vision-based perception limitations (e.g.…”

“…Email: {potena, nardi}@diag.uniroma1.it. 2 Pretto is with IT+Robotics S.r.l., Vicenza, Italy. Email: alberto.pretto@it-robotics.eu Fig.…”