In the past two decades, unmanned aerial vehicles (UAVs) have demonstrated their efficacy in supporting both military and civilian applications, where tasks can be dull, dirty, dangerous, or simply too costly with conventional methods. Many of the applications contain tasks that can be executed in parallel, hence the natural progression is to deploy multiple UAVs working together as a force multiplier. However, to do so requires autonomous coordination among the UAVs, similar to swarming behaviors seen in animals and insects. This paper looks at flocking with small fixed-wing UAVs in the context of a model-free reinforcement learning problem. In particular, Peng's Q(λ) with a variable learning rate is employed by the followers to learn a control policy that facilitates flocking in a leader-follower topology. The problem is structured as a Markov decision process, where the agents are modeled as small fixed-wing UAVs that experience stochasticity due to disturbances such as winds and control noises, as well as weight and balance issues. Learned policies are compared to ones solved using stochastic optimal control (i.e., dynamic programming) by evaluating the average cost incurred during flight according to a cost function. Simulation results demonstrate the feasibility of the proposed learning approach at enabling agents to learn how to flock in a leader-follower topology, while operating in a nonstationary stochastic environment.

No abstract

In order for teams of unmanned aerial vehicles (UAVs) to collaborate and cooperate to perform challenging group tasks, intelligent and flexible control strategies are required. One of the complex behaviors required of a team of UAVs is dynamic encirclement, which is a tactic that can be employed for persistent surveillance and/or to neutralize a target by restricting its movement. This tactic requires a high level of cooperation such that the UAVs maintain a desired and proper encirclement radius and angular velocity around the target. In this paper, model predictive control (MPC) is used to model and implement controllers for the problem of dynamic encirclement. The linear and nonlinear control policies proposed in this paper are applied as a high-level controller to control multiple UAVs to encircle a desired target in simulations and real-time experiments with quadrotors. The nonlinear solution provides a theoretical analysis of the problem, while the linear control policy is used for real-time operation via a combination of MPC and feedback linearization applied to the nonlinear UAV system. The contributions of this paper lie in the implementation of MPC to solve the problem of dynamic encirclement of a team of UAVs in real time and the application of theoretical stability analysis to the problem.

Recent robotics applications require 3-D representations of the environments. In many cases, it is not feasible for a single robot to map the entire environment. In these cases, it is necessary for a team of robots to build maps independently and merge them into a single global map. In this paper, octree-based occupancy grids, which are currently the state-of-the-art 3-D map representation, are applied to the problem of multirobot mapping. Octrees allow large environments to be mapped efficiently, in terms of memory usage, while still providing sufficiently fine resolution where required. The main contribution of this work lies in the definition and validation of a system, which use map data from commonly mapped portions of the environment with registration techniques, such that maps are merged coherently despite measurement noise and error in the relative transformations between maps for experimental data sets. The system defined can then be used in a complete solution that is ported to mobile robots. The results demonstrate that octree occupancy grids are a suitable representation for multirobot 3-D mapping, but that the proposed techniques for improving erroneous transformation estimates between map frames allow multiple maps to be merged efficiently and robustly.

No abstract