Study of Q-learning and deep Q-network learning control for a rotary inverted pendulum system

In general, the trajectory tracking of robotic manipulator is exceptionally challenging due to the complex and strongly coupled mechanical architecture. In this paper, precise track control of the robotic manipulator is formulated as a dense reward problem for reinforcement learning(RL). A deep RL(DRL) approach combining the soft actor-critic (SAC) algorithm and ensemble random network distillation (ERND) is proposed to address the tracking control problem for robotic manipulator. Firstly, an ERND model is designed, consisting of a module list of multiple RND models. Each RND model obtains the error by learning the target features and the predicted features of the environment. The resulting error serves as internal rewards that drive the robotic agent to explore unknown and unpredictable environmental states. The ensemble model obtains the total internal reward by summing the internal rewards of each RND model, thereby obtaining more accurately reflecting the characteristics of the manipulator in tracking control tasks and improving control performance. Secondly, combining the SAC algorithm with ERND facilitates more robust exploration capabilities in environments with input saturation and joint angle constraints, thereby enabling faster learning of effective policies and enhancing the performance and efficiency of robotic manipulator tracking control tasks. Finally, the simulation results demonstrate that the robotic manipulator tracking control task is effectively completed in dense reward problems through the combination of the SAC algorithm and ERND.

show abstract

A Navigation Algorithm Based on the Reinforcement Learning Reward System and Optimised with Genetic Algorithm

Cabezas-Olivenza,

Zulueta,

Azurmendi-Marquinez

et al. 2024

Mathematics

View full text Add to dashboard Cite

Regarding autonomous vehicle navigation, reinforcement learning is a technique that has demonstrated significant results. Nevertheless, it is a technique with a high number of parameters that need to be optimised without prior information, and correctly performing this is a complicated task. In this research study, a system based on the principles of reinforcement learning, specifically on the concept of rewards, is presented. A mathematical expression was proposed to control the vehicle’s direction based on its position, the obstacles in the environment and the destination. In this equation proposal, there was only one unknown parameter that regulated the degree of the action to be taken, and this was optimised through the genetic algorithm. In this way, a less computationally expensive navigation algorithm was presented, as it avoided the use of neural networks. The controller’s time to obtain the navigation instructions was around 6.201 10−4 s. This algorithm is an efficient and accurate system which manages not to collide with obstacles and to reach the destination from any position. Moreover, in most cases, it has been found that the proposed navigations are also optimal.

show abstract

Study of Q-learning and deep Q-network learning control for a rotary inverted pendulum system

Cited by 4 publications

References 28 publications

Study of Inverse Kinematics Solution for a 5-Axis Mitsubishi RV-2AJ Robotic Arm Using Deep Reinforcement Learning

Study of Inverse Kinematics Solution for a 5-Axis Mitsubishi RV-2AJ Robotic Arm Using Deep Reinforcement Learning

Trajectory tracking control based on deep reinforcement learning and ensemble random network distillation for robotic manipulator

A Navigation Algorithm Based on the Reinforcement Learning Reward System and Optimised with Genetic Algorithm

Contact Info

Product

Resources

About