Developmental Reinforcement Learning of Control Policy of a Quadcopter UAV With Thrust Vectoring Rotors

Abstract

In this paper, we present a novel developmental reinforcement learning-based controller for a quadcopter with thrust vectoring capabilities. This multirotor UAV design has tilt-enabled rotors. It utilizes the rotor force magnitude and direction to achieve the desired state during flight. The control policy of this robot is learned using the policy transfer from the learned controller of the quadcopter (comparatively simple UAV design without thrust vectoring). This approach allows learning a control policy for systems with multiple inputs and multiple outputs. The performance of the learned policy is evaluated by physics-based simulations for the tasks of hovering and way-point navigation. The flight simulations utilize a flight controller based on reinforcement learning without any additional PID components. The results show faster learning with the presented approach as opposed to learning the control policy from scratch for this new UAV design created by modifications in a conventional quadcopter, i.e., the addition of more degrees of freedom (4-actuators in conventional quadcopter to 8-actuators in tilt-rotor quadcopter). We demonstrate the robustness of our learned policy by showing the recovery of the tilt-rotor platform in the simulation from various non-static initial conditions in order to reach a desired state. The developmental policy for the tilt-rotor UAV also showed superior fault tolerance when compared with the policy learned from the scratch. The results show the ability of the presented approach to bootstrap the learned behavior from a simpler system (lower-dimensional action-space) to a more complex robot (comparatively higher-dimensional action-space) and reach better performance faster.

Data‐enabled predictive control for quadcopters

“…In particular, we seek to analyze how the algorithm can be applied for real‐time control of a quadcopter whose dynamics are nonlinear and the measurements are corrupted by noise. The quadcopter is a common benchmark system for verifying data‐driven control methods 7,8,28‐30 . It makes for an interesting benchmark because it is nonlinear, open‐loop unstable and has fast dynamics.…”

Enhancement of Drones’ Control and Guidance Systems Channels: A Review

“…This results in the drone ascending or descending. The remaining three channels (referred to as the aileron, elevator, and rudder in radio control) control the roll, pitch, and yaw axes, respectively [2,3]. These three control inputs operate by altering the attitude of the quad-rotor (tilt or direction).…”

“…The problem of under-actuation is that a quad-rotor is a six degreeof-freedom flying vehicle, but has only four actuations (variable rotors lift due to their variable RPM). This shortage is treated by increasing the actuations thru adding an electric ducted fan (4 thrusters) control design approach, which increases the actuations of the quad-rotor from four to eight [3].…”

“…In which we will analyses thruster's novel concepts from two points of view. First, is the multi-rotor concept design (vehicle airframe) and second the mathematical model of Thruster Quad-rotor drone for analyzing its control and guidance systems channels [3].…”

A novel approach to control four multi-rotor drones in cooperative paired control using relative Jacobian

“…This paper addresses the problem faced with the usual approach in controlling multi-rotor drones to maneuver in 3D space using tilting heuristics by running the rotors at different speeds as seen in Fig. 1 [14][15][16][17].…”