Contemporary autopilot systems for unmanned aerial vehicles (UAVs) are far more limited in their flight envelope as compared to experienced human pilots, thereby restricting the conditions UAVs can operate in and the types of missions they can accomplish autonomously. This paper proposes a deep reinforcement learning (DRL) controller to handle the nonlinear attitude control problem, enabling extended flight envelopes for fixed-wing UAVs. A proof-of-concept controller using the proximal policy optimization (PPO) algorithm is developed, and is shown to be capable of stabilizing a fixed-wing UAV from a large set of initial conditions to reference roll, pitch and airspeed values. The training process is outlined and key factors for its progression rate are considered, with the most important factor found to be limiting the number of variables in the observation vector, and including values for several previous time steps for these variables. The trained reinforcement learning (RL) controller is compared to a proportional-integralderivative (PID) controller, and is found to converge in more cases than the PID controller, with comparable performance. Furthermore, the RL controller is shown to generalize well to unseen disturbances in the form of wind and turbulence, even in severe disturbance conditions.

This paper presents a model of an electrical propulsion system typically used for small fixed wing unmanned aerial vehicles (UAVs). Such systems consist of a power source, an electronic speed controller and a brushless DC motor which drives a propeller. The electrical, mechanical and aerodynamic subsystems are modeled separately and then combined into one system model, aiming at bridging the gap between the more complex models used in manned aviation and the simpler models typically used for UAVs. Such a model allows not only the prediction of thrust but also of the propeller speed and consumed current. This enables applications such as accurate range and endurance estimation, UAV simulation and modelbased control, in-flight aerodynamic drag estimation and propeller icing detection. Wind tunnel experiments are carried out to validate the model, which is also compared to two UAV propulsion models found in the literature. The experimental results show that the model is able to predict thrust well, with a root mean square error (RMSE) of 2.20 percent of max thrust when RPM measurements are available, and an RMSE of 4.52 percent without.

This paper presents nonlinear, singularity-free autopilot designs for multivariable reduced-attitude control of fixed-wing aircraft. To control roll and pitch angles, we employ vector coordinates constrained to the unit two-sphere and that are independent of the yaw/heading angle. The angular velocity projected onto this vector is enforced to satisfy the coordinated-turn equation. We exploit model structure in the design and prove almost global asymptotic stability using Lyapunov-based tools. Slowly-varying aerodynamic disturbances are compensated for using adaptive backstepping. To emphasize the practical application of our result, we also establish the ultimate boundedness of the solutions under a simplified controller that only depends on rough estimates of the control-effectiveness matrix. The controller design can be used with state-of-the-art guidance systems for fixed-wing unmanned aerial vehicles (UAVs) and is implemented in the open-source autopilot ArduPilot for validation through realistic software-in-the-loop (SITL) simulations.

Attitude control of fixed-wing unmanned aerial vehicles (UAVs) is a difficult control problem in part due to uncertain nonlinear dynamics, actuator constraints, and coupled longitudinal and lateral motions. Current state-of-the-art autopilots are based on linear control and are thus limited in their effectiveness and performance. Deep reinforcement learning (DRL) is a machine learning method to automatically discover optimal control laws through interaction with the controlled system, that can handle complex nonlinear dynamics. We show in this paper that DRL can successfully learn to perform attitude control of a fixedwing UAV operating directly on the original nonlinear dynamics, requiring as little as three minutes of flight data. We initially train our model in a simulation environment and then deploy the learned controller on the UAV in flight tests, demonstrating comparable performance to the state-of-the-art ArduPlane proportional-integral-derivative (PID) attitude controller with no further online learning required. To better understand the operation of the learned controller we present an analysis of its behaviour, including a comparison to the existing well-tuned PID controller.

Inner-loop control algorithms in state-of-the-art autopilots for fixed-wing unmanned aerial vehicles (UAVs) are typically designed using linear control theory, to operate in relatively conservative flight envelopes. In the Autofly project, we seek to extend the flight envelopes of fixed-wing UAVs to allow for more aggressive maneuvering and operation in a wider range of weather conditions. Throughout the last few years, we have successfully flight tested several inner-loop attitude controllers for fixed-wing UAVs using advanced nonlinear control methods, including nonlinear model predictive control (NMPC), deep reinforcement learning (DRL), and geometric attitude control. To achieve this, we have developed a flexible embedded platform, capable of running computationally demanding lowlevel controllers that require direct actuator control. For safe operation and rapid development cycles, this platform can be deployed in tandem with well-tested standard autopilots. In this paper, we summarize the challenges and lessons learned, and document the system architecture of our experimental platform in a best-practice manner. This lowers the threshold for other researchers and engineers to employ new low-level control algorithms for fixed-wing UAVs. Case studies from outdoor field experiments are provided to demonstrate the efficacy of our research platform.