Vision-Based Reactive Planning for Aggressive Target Tracking While Avoiding Collisions and Occlusions

Penin, Bryan; Giordano, Paolo Robuffo; Chaumette, François

doi:10.1109/lra.2018.2856526

Cited by 97 publications

(77 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed model is general enough to be valid in all these situations. Grouping together (5), (6) and (4), the dynamics of the multirotor is expressed as a set of nonlinear differential equations, synthetically denoted in the following aṡ…”

Section: A Equality Constraints Due To the Multi-rotor Dynamicsmentioning

confidence: 99%

“…It is assumed to have a sensor S rigidly mounted on the UAV such that the transformation between F S -the sensor frame -and F B is constant and known. Therefore, the perception constraints can be expressed as inequalities [4] which are functions of x b and some sensor parameters:…”

Section: Inequality Constraints Induced By Perceptionmentioning

confidence: 99%

“…In particular we focus our attention here on a common application of perceptive MPC, i.e., the feature-covering along a motion [2]- [4], [14]. The sensor S is assumed to be able to perceive a feature in the environment only if it falls inside a conic Field of View (FoV) with principal axis z S and halfwidth equal to α.…”

Section: Inequality Constraints Induced By Perceptionmentioning

confidence: 99%

See 2 more Smart Citations

Perception-constrained and Motor-level Nonlinear MPC for both Underactuated and Tilted-propeller UAVS

Jacquet

Corsini

Bicego

et al. 2020

2020 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

In this paper, we present a Perception-constrained Nonlinear Model Predictive Control (NMPC) framework for the real-time control of multi-rotor aerial vehicles. Our formulation considers both constraints from a perceptive sensor and realistic actuator limitations that are the rotor minimum and maximum speeds and accelerations. The formulation is meant to be generic and considers a large range of multi-rotor platforms (such as underactuated quadrotors or tilted-propellers hexarotors) since it does not rely on differential flatness for the dynamical equations, and a broad range of sensors, such as cameras, lidars, etc... The perceptive constraints are expressed to maintain visibility of a feature point in the sensor's field of view, while performing a reference maneuver. We demonstrate both in simulation and real experiments that our framework is able to exploit the full capabilities of the multi-rotor, to achieve the motion under the aforementioned constraints, and control in real-time the platform at a motor-torque level, avoiding the use of an intermediate unconstrained trajectory tracker.

show abstract

Section: A Equality Constraints Due To the Multi-rotor Dynamicsmentioning

confidence: 99%

Section: Inequality Constraints Induced By Perceptionmentioning

confidence: 99%

Section: Inequality Constraints Induced By Perceptionmentioning

confidence: 99%

See 1 more Smart Citation

Perception-constrained and Motor-level Nonlinear MPC for both Underactuated and Tilted-propeller UAVS

Jacquet

Corsini

Bicego

et al. 2020

2020 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…In a previous work [14] we considered hard visibility constraints that may become too restrictive for minimum-time planning. In this work, we propose to relax these constraints by allowing temporary visual cues losses for performing faster trajectories in larger and more complex environments.…”

Section: B Contributionsmentioning

confidence: 99%

Minimum-Time Trajectory Planning Under Intermittent Measurements

Penin

Giordano

Chaumette

2019

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

This paper focuses on finding robust paths for a robotic system by taking into account the state uncertainty and the probability of collision. We are interested in dealing with intermittent exteroceptive measurements (e.g., collected from vision). We assume that these cues provide reliable measurements that will update a state estimation algorithm wherever they are available. The planner has to manage two tasks: reaching the goal in a minimum time and collecting sufficient measurements to reach the goal state with a given confidence level. We present a robust perception-aware bi-directional A* planner for differentially flat systems such as a unicycle and a quadrotor UAV and use a derivative-free Kalman filter to approximate the belief dynamics in the flat space. We also propose an efficient way of ensuring continuity and feasibility by exploiting the convex-hull property of B-spline curves.

show abstract

“…When dealing with dynamic targets, the UAV must anticipate the actor's motion and reason about collisions and occlusions generated by obstacles in potential trajectories. The challenge of visibility has been explored in previous works in varying degrees of obstacle complexity (Bonatti, Ho, Wang, Choudhury, & Scherer, ; Galvane et al, ; Nägeli et al, ; Penin, Giordano, & Chaumette, ).…”

Section: Introductionmentioning

confidence: 99%

Autonomous aerial cinematography in unstructured environments with learned artistic decision‐making

Bonatti

Wang

et al. 2020

Journal of Field Robotics

View full text Add to dashboard Cite

Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often requiring multiple expert human operators. Hence, there is a demand for an autonomous cinematographer that can reason about both geometry and scene context in real-time. Existing approaches do not address all aspects of this problem; they either require high-precision motion-capture systems or global positioning system tags to localize targets, rely on prior maps of the environment, plan for short time horizons, or only follow fixed artistic guidelines specified before the flight. In this study, we address the problem in its entirety and propose a complete system for real-time aerial cinematography that for the first time combines: (a) vision-based target estimation; (b) 3D signed-distance mapping for occlusion estimation; (c) efficient trajectory optimization for long time-horizon camera motion; and (d) learning-based artistic shot selection.We extensively evaluate our system both in simulation and in field experiments by filming dynamic targets moving through unstructured environments. Our results indicate that our system can operate reliably in the real world without restrictive assumptions. We also provide in-depth analysis and discussions for each module, with the hope that our design tradeoffs can generalize to other related applications. Videos of the complete system can be found at https://youtu.be/ookhHnqmlaU. K E Y W O R D S aerial robotics, cinematography, computer vision, learning, mapping, motion planning Within the filming context, this cost function measures jerkiness of motion, safety, environmental occlusion of the actor, and shot quality (artistic quality of viewpoints). This cost function depends on the environment and , and on the actor forecast ξ a , all of which are sensed on-the-fly. The changing nature of the environment and ξ a demands replanning at a high frequency. Here we briefly touch upon the four components of the cost function J(ξ q ) (refer to Section 7 for details and mathematical expressions): Smoothness J smooth (ξ q ): Penalizes jerky motions that may lead to camera blur and unstable flight; Safety J obs (ξ q , ): Penalizes proximity to obstacles that are unsafe for the UAV; Occlusion J occ (ξ q , ξ a , ): Penalizes occlusion of the actor by obstacles in the environment; Shot quality J shot (ξ q , ξ a , Ω art ): Penalizes poor viewpoint angles and scales that deviate from the desired artistic guidelines, given by the set of parameters Ω art . In its simplest form, we can express J(ξ q ) as a linear composition of each individual cost, weighted by scalars λ i . The objective is to * = ( ) ()={ } J J J J J J xyz J J J J J J x y z J occ + J obs 99.4 ± 2.2 94.2 ± 7.3 86.9 ± 9.3 J obs 98.8 ± 3.0 87.1 ± 8.5 75.3 ± 11.8 Avg. dist. to ξ shot (m) J occ + J obs 0.4 ± 0.4 6.2 ± 11.2 10.7 ± 13.2 J obs 0.05 ± 0.1 0.3 ± 0.2 0.5 ± 0...

show abstract

Vision-Based Reactive Planning for Aggressive Target Tracking While Avoiding Collisions and Occlusions

Cited by 97 publications

References 34 publications

Perception-constrained and Motor-level Nonlinear MPC for both Underactuated and Tilted-propeller UAVS

Perception-constrained and Motor-level Nonlinear MPC for both Underactuated and Tilted-propeller UAVS

Minimum-Time Trajectory Planning Under Intermittent Measurements

Autonomous aerial cinematography in unstructured environments with learned artistic decision‐making

Contact Info

Product

Resources

About