Reinforcement learning based flight controller capable of controlling a quadcopter with four, three and two working motors

Abstract

Curiosity is one of the main motives in many of the natural creatures with measurable levels of intelligence for exploration and, as a result, more efficient learning. It makes it possible for humans and many animals to explore efficiently by searching for being in states that make them surprised with the goal of learning more about what they do not know. As a result, while being curious, they learn better. In the machine learning literature, curiosity is mostly combined with reinforcement learning-based algorithms as an intrinsic reward. This work proposes an algorithm based on the drive of curiosity for autonomous learning to control by generating proper motor speeds from odometry data. The quadcopter controlled by our proposed algorithm can pass through obstacles while controlling the Yaw direction of the quad-copter toward the desired location. To achieve that, we also propose a new curiosity approach based on prediction error. We ran tests using on-policy, off-policy, on-policy plus curiosity, and the proposed algorithm and visualized the effect of curiosity in evolving exploration patterns. Results show the capability of the proposed algorithm to learn optimal policy and maximize reward where other algorithms fail to do so.

Genetically Tuned Linear Quadratic Regulator for Trajectory Tracking of a Quadrotor

“…Reinforcement learning techniques are also used, which iteratively perform learning processes independently of the system model. In fact, there are studies to carry out the control process with those that remain intact against engine failures that occur in the aircraft with these techniques [12]. However, reinforcement learning processes cannot guarantee that the system will always produce stable responses.…”

An Adaptive PID Autotuner for Multicopters with Experimental Results

“…Iterative learning control is used learn the pitch and roll controller in the closed-loop system in [12]. Reinforcement learning control is used to learn low-level controllers in case of multiple actuator failure in [13]. L1 adaptive control is used to improve the stability margins of a stable control loop in a quadcopter flight control system in [14].…”