Model-Free Attitude Control of Spacecraft Based on PID-Guide TD3 Algorithm

Zhang, Zhibin; Li, Xinhong; An, Jiping; Man, Wanxin; Zhang, Guohui

doi:10.1155/2020/8874619

Cited by 23 publications

(4 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [ 33 ], an end-to-end automatic lane changing method was proposed for autonomous vehicles using the DDPG algorithm. In [ 34 ], a Proportional–Integral–Derivative (PID)-Guide controller was designed to continuously learn through RL according to the feedback of environment to achieve high-precision attitude control of spacecraft. In [ 35 ], a controller based on the Robust-DDPG algorithm was developed for UAVs to fly stably in uncertain environments; the controller can continuously control two desired variables (roll and speed) of the UAV.…”

Section: Related Workmentioning

confidence: 99%

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

Dai

Chen

Yang

2022

Sensors

View full text Add to dashboard Cite

Using reinforcement learning (RL) for torque distribution of skid steering vehicles has attracted increasing attention recently. Various RL-based torque distribution methods have been proposed to deal with this classical vehicle control problem, achieving a better performance than traditional control methods. However, most RL-based methods focus only on improving the performance of skid steering vehicles, while actuator faults that may lead to unsafe conditions or catastrophic events are frequently omitted in existing control schemes. This study proposes a meta-RL-based fault-tolerant control (FTC) method to improve the tracking performance of vehicles in the case of actuator faults. Based on meta deep deterministic policy gradient (meta-DDPG), the proposed FTC method has a representative gradient-based metalearning algorithm workflow, which includes an offline stage and an online stage. In the offline stage, an experience replay buffer with various actuator faults is constructed to provide data for training the metatraining model; then, the metatrained model is used to develop an online meta-RL update method to quickly adapt its control policy to actuator fault conditions. Simulations of four scenarios demonstrate that the proposed FTC method can achieve a high performance and adapt to actuator fault conditions stably.

show abstract

Section: Related Workmentioning

confidence: 99%

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

Dai

Chen

Yang

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…PIDnn Controller Design. PIDnn is a kind of PID-type controller that relies on the self-adaptation and learning ability of the neural network algorithm [25]. There are various neural network structures that can be designed.…”

Section: System Dynamics Modelmentioning

confidence: 99%

3-DOF Position and Orientation Control of an Air Flotation Platform for Spacecraft Ground Microgravity Simulation by Using Double Closed-Loop Cascade PIDnn

Wang

Gao

Liu

et al. 2022

International Journal of Aerospace Engineering

View full text Add to dashboard Cite

Space assistant robots can help astronauts or their assistants perform certain tasks. A ground microgravity simulation environment is built for the space assistant robot AAR-2. The hardware requirements of the ground simulation by the 3-DOF microgravity air flotation platform. An algorithm is designed for this simulation system. By using momentum and RMSprop methods to improve the PID neural network, the challenging problem of strong coupling between system nonlinearity and variables is solved. Firstly, the paper introduces the hardware system and deduces the dynamic model of the system. Then, the algorithm is calculated and simulated. Through simulation, the effectiveness and feasibility of the algorithm are compared and proved. Finally, the control system is simulated by MATLAB/Simulink and compared with other advanced algorithms. The simulation results show that the designed neural network controller can quickly and accurately control the 3-DOF of freedom motion of AAR-2.

show abstract

“…The deep learning nature of DDPG may allow autonomous operations, if the network configuration, its hyperparameters, and the reward function are carefully designed. There are many studies focused on implementing DDPG in different environments and/or improving its performance by modifying the algorithm [15][16][17][18][19]. Specifically, DDPG is deployed in the trajectory planning of a dual-arm robot that provides on-orbit services [15].…”

Section: Introductionmentioning

confidence: 99%

Trajectory Generation for Space Manipulators Capturing Moving Targets Using Transfer Learning

Sze¹,

Chhabra²

2023

Preprint

View full text Add to dashboard Cite

<p>In a debris mitigation mission, a crucial phase of the proximity operation for a space manipulator is chasing a capture point on a noncooperative target satellite. Knowing the uncertain position and velocity of the target, a learning-based online trajectory planner offers a robust solution to the chasing problem. This paper uses the concept of transfer learning to develop an online trajectory generator for the task of capturing a moving target with an uncertain space manipulator. We divide this complex task into multiple sub-tasks and order them based on their difficulty level. We employ the Deep Deterministic Policy Gradient (DDPG) algorithm to learn each sub-task individually. The DDPG is a deep reinforcement learning approach that provides the ability to work with continuous states and actions by approximating the action-value function and the policy with neural networks. We propose a novel method to transfer the knowledge gained in an easier sub-task to a more difficult one in the form of expert policy and transition memories. State and action representation has a crucial impact on learning performance, which is comprehensively studied in this paper for the task of capturing a moving target. Considering the learning performance, we show the existence of an optimal state representation, which is not necessarily the minimal representation of the system. We compare different action representations of a manipulator, i.e., joint space and workspace velocities, and demonstrate the superiority of the workspace actions. Finally, the developed transfer learning approach is implemented on a planar space manipulator with an onboard 2-link arm to generate trajectories that can capture a target randomly moving with the maximum speed of the manipulator’s end effector. To show the efficacy of the approach, its results are compared with the case where the agent learns the task from scratch.</p>

show abstract

Model-Free Attitude Control of Spacecraft Based on PID-Guide TD3 Algorithm

Cited by 23 publications

References 23 publications

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

Metalearning-Based Fault-Tolerant Control for Skid Steering Vehicles under Actuator Fault Conditions

3-DOF Position and Orientation Control of an Air Flotation Platform for Spacecraft Ground Microgravity Simulation by Using Double Closed-Loop Cascade PIDnn

Trajectory Generation for Space Manipulators Capturing Moving Targets Using Transfer Learning

Contact Info

Product

Resources

About