A PID Gain Adjustment Scheme Based on Reinforcement Learning Algorithm for a Quadrotor

Zheng, Quan; Tang, Renjie; Gou, Siyuan; Zhang, Weizhong

doi:10.23919/ccc50068.2020.9188426

Cited by 5 publications

(7 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The state space vector X describes the position of the quadcopter in space and its linear and angular velocities as follows [5][6][7][8][17][18][19][20]:…”

Section: Quadcopter State Space Modelmentioning

confidence: 99%

“…The quadcopter's orientation is represented by the three Euler angles: φ represents the roll angle around the x-axis, θ represents the pitch angle around the y-axis, and ψ represents the yaw angle around the z-axis [5][6][7]. In order to obtain the state space representation of the quadcopter, the following equations that describe the translational and rotational motion of the quadcopter are used [17][18][19][20]:…”

Section: Quadcopter State Space Modelmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing Quadcopter Autonomy: Implementing Advanced Control Strategies and Intelligent Trajectory Planning

Hadid,

Boushaki,

Boumchedda

et al. 2024

Automation

View full text Add to dashboard Cite

In this work, an in-depth investigation into enhancing quadcopter autonomy and control capabilities is presented. The focus lies on the development and implementation of three conventional control strategies to regulate the behavior of quadcopter UAVs: a proportional–integral–derivative (PID) controller, a sliding mode controller, and a fractional-order PID (FOPID) controller. Utilizing careful adjustments and fine-tuning, each control strategy is customized to attain the desired dynamic response and stability during quadcopter flight. Additionally, an approach called Dyna-Q learning for obstacle avoidance is introduced and seamlessly integrated into the control system. Leveraging MATLAB as a powerful tool, the quadcopter is empowered to autonomously navigate complex environments, adeptly avoiding obstacles through real-time learning and decision-making processes. Extensive simulation experiments and evaluations, conducted in MATLAB 2018a, precisely compare the performance of the different control strategies, including the Dyna-Q learning-based obstacle avoidance technique. This comprehensive analysis allows us to understand the strengths and limitations of each approach, guiding the selection of the most effective control strategy for specific application scenarios. Overall, this research presents valuable insights and solutions for optimizing flight stability and enabling secure and efficient operations in diverse real-world scenarios.

show abstract

“…The state space vector X describes the position of the quadcopter in space and its linear and angular velocities as follows [5][6][7][8][17][18][19][20]:…”

Section: Quadcopter State Space Modelmentioning

confidence: 99%

Section: Quadcopter State Space Modelmentioning

confidence: 99%

Enhancing Quadcopter Autonomy: Implementing Advanced Control Strategies and Intelligent Trajectory Planning

Hadid,

Boushaki,

Boumchedda

et al. 2024

Automation

View full text Add to dashboard Cite

show abstract

“…Combining it with the traditional PID algorithm can improve the generalization and adaptability of a PID controller. Zheng proposed a PID controller based on reinforcement learning to enhance the trajectory tracking performance in a quadrotor [29]. Combining the proximal policy optimization (PPO) algorithm with the traditional PID controller, this method has improved response time, reduced overshoot, minimized control errors, enhanced stability, and strong anti-interference capabilities compared with the traditional PID controller.…”

Section: Introductionmentioning

confidence: 99%

Velocity Control of a Multi-Motion Mode Spherical Probe Robot Based on Reinforcement Learning

Cao³

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

As deep space exploration tasks become increasingly complex, the mobility and adaptability of traditional wheeled or tracked probe robots with high functional density are constrained in harsh, dangerous, or unknown environments. A practical solution to these challenges is designing a probe robot for preliminary exploration in unknown areas, which is characterized by robust adaptability, simple structure, light weight, and minimal volume. Compared to the traditional deep space probe robot, the spherical robot with a geometric, symmetrical structure shows better adaptability to the complex ground environment. Considering the uncertain detection environment, the spherical robot should brake rapidly after jumping to avoid reentering obstacles. Moreover, since it is equipped with optical modules for deep space exploration missions, the spherical robot must maintain motion stability during the rolling process to ensure the quality of photos and videos captured. However, due to the nonlinear coupling and parameter uncertainty of the spherical robot, it is tedious to adjust controller parameters. Moreover, the adaptability of controllers with fixed parameters is limited. This paper proposes an adaptive proportion–integration–differentiation (PID) control method based on reinforcement learning for the multi-motion mode spherical probe robot (MMSPR) with rolling and jumping. This method uses the soft actor–critic (SAC) algorithm to adjust the parameters of the PID controller and introduces a switching control strategy to reduce static error. As the simulation results show, this method can facilitate the MMSPR’s convergence within 0.02 s regarding motion stability. In addition, in terms of braking, it enables an MMSPR with random initial speed brake within a convergence time of 0.045 s and a displacement of 0.0013 m. Compared with the PID method with fixed parameters, the braking displacement of the MMSPR is reduced by about 38%, and the convergence time is reduced by about 20%, showing better universality and adaptability.

show abstract

“…Through training and testing in the RotorS-Gazebo environment, it was proved that the tracking performance of the method was better than that of the NLGL (Nonlinear Navigation Logic) method. Zheng Q et al [27] used the PPO algorithm of reinforcement learning to adjust the PID controller gain, and achieved good stability of the aircraft in control, anti-jamming and flying height. In addition, Zhen Y et al [28,29] proposed a hybrid DDPG (Mi-DDPG) algorithm.…”

Section: Introductionmentioning

confidence: 99%

A State-Compensated Deep Deterministic Policy Gradient Algorithm for UAV Trajectory Tracking

Yang

Liao

et al. 2022

Machines

View full text Add to dashboard Cite

The unmanned aerial vehicle (UAV) trajectory tracking control algorithm based on deep reinforcement learning is generally inefficient for training in an unknown environment, and the convergence is unstable. Aiming at this situation, a Markov decision process (MDP) model for UAV trajectory tracking is established, and a state-compensated deep deterministic policy gradient (CDDPG) algorithm is proposed. An additional neural network (C-Net) whose input is compensation state and output is compensation action is added to the network model of a deep deterministic policy gradient (DDPG) algorithm to assist in network exploration training. It combined the action output of the DDPG network with compensated output of the C-Net as the output action to interact with the environment, enabling the UAV to rapidly track dynamic targets in the most accurate continuous and smooth way possible. In addition, random noise is added on the basis of the generated behavior to realize a certain range of exploration and make the action value estimation more accurate. The OpenAI Gym tool is used to verify the proposed method, and the simulation results show that: (1) The proposed method can significantly improve the training efficiency by adding a compensation network and effectively improve the accuracy and convergence stability; (2) Under the same computer configuration, the computational cost of the proposed algorithm is basically the same as that of the QAC algorithm (Actor-critic algorithm based on behavioral value Q) and the DDPG algorithm; (3) During the training process, with the same tracking accuracy, the learning efficiency is about 70% higher than that of QAC and DDPG; (4) During the simulation tracking experiment, under the same training time, the tracking error of the proposed method after stabilization is about 50% lower than that of QAC and DDPG.

show abstract

A PID Gain Adjustment Scheme Based on Reinforcement Learning Algorithm for a Quadrotor

Cited by 5 publications

References 10 publications

Enhancing Quadcopter Autonomy: Implementing Advanced Control Strategies and Intelligent Trajectory Planning

Enhancing Quadcopter Autonomy: Implementing Advanced Control Strategies and Intelligent Trajectory Planning

Velocity Control of a Multi-Motion Mode Spherical Probe Robot Based on Reinforcement Learning

A State-Compensated Deep Deterministic Policy Gradient Algorithm for UAV Trajectory Tracking

Contact Info

Product

Resources

About