PPO-Based Attitude Controller Design for a Tilt Rotor UAV in Transition Process

Abstract

The complex aerodynamic changes of the tilt-rotor UAV (TRUAV) in the transition process show strong nonlinearity, which brings a great impact on the stability of the vehicle attitude. This study aims to design a PPO-based RL controller for attitude control in the transition process. A reinforcement-learning PPO approach is used to learn the control strategy by interacting directly with the environment. And the reward function is designed and improved for the transition process. The performance of the proposed controller is tested and compared by simulation. The results show that the PPO algorithm is more suitable for the tilt-rotor transition process control than the A2C algorithm. Our proposed reward function improves the attitude control performance and the designed RL controller has good adaptability to changes in the takeoff weight, the diagonal wheelbase and the tilt rate. This study highlights the effectiveness and potential of reinforcement learning for tilt-rotor UAV transition process attitude control. These findings contribute to the advancement of autonomous flight systems by providing insights into the application of reinforcement learning algorithms. These results have important implications for the development of intelligent flight control systems and could guide future research in this area.

A Deep Learning Approach for Trajectory Control of Tilt-Rotor UAV

“…The resulting controller is subsequently evaluated within the same simulated environment, exhibiting its ability to achieve satisfactory closedloop performance. Further works presented in references [24][25][26] elaborate on the potential and capabilities of deep learning methodologies for constructing various types of flight control systems for UAVs, enabling the attainment of desired performance objectives, such as stability and accurate tracking, across various flight phases, including the transition phase for tilt-rotor UAVs.…”

“…Some of the works on machine learning control for multi-copter systems [23,24], and also for tilt-rotor systems [25,26], employ a reinforced learning approach, where the learning process is regulated using the reward function and cumulative cost value. In those reinforced schemes, a configuration called the actor-critic network is utilized, where the input-output variables of the controller model are initially defined, requiring a priori comprehensive knowledge about the behavior of the controlled system.…”

“…In those reinforced schemes, a configuration called the actor-critic network is utilized, where the input-output variables of the controller model are initially defined, requiring a priori comprehensive knowledge about the behavior of the controlled system. The networks are trained using simulated data that is generated using Gazebo simulation software [23,26] or the more accomplished digital twin environment [24,25].…”