Fuzzy Double Deep Q-Network-Based Gait Pattern Controller for Humanoid Robots

Li, Tzuu-Hseng S.; Kuo, Ping‐Huan; Chen, Lin-Han; Hung, Chia-Ching; Luan, Po‐Chien; Hsu, Hao-Ping; Chang, Chien-Hsin; Hsieh, Yi‐Ting; Lin, Wen-Hsun

doi:10.1109/tfuzz.2020.3033141

Cited by 13 publications

(3 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To meet control demands and ensure that the robot's speed, displacement, foot placement and other variables of the robot are consistent with the desired trajectory, it is necessary to further study the motion-tracking control of the joint. A variety of control methods have been developed to address the motion trajectory tracking problem, such as PD control [13,14,15], fuzzy control [16,17,18], neural network [19,20,21,22,23,24], model predictive control (MPC) [25,26], human-simulated intelligent control (HSIC) [27,28] or a variety of combination methods [29,30,31,32,33].…”

Section: Introductionmentioning

confidence: 99%

A human-simulated fuzzy membrane approach for the joint controller of walking biped robots

Liu

Zhang

Mastoi

et al. 2023

ICA

View full text Add to dashboard Cite

To guarantee their locomotion, biped robots need to walk stably. The latter is achieved by a high performance in joint control. This article addresses this issue by proposing a novel human-simulated fuzzy (HF) membrane control system of the joint angles. The proposed control system, human-simulated fuzzy membrane controller (HFMC), contains several key elements. The first is an HF algorithm based on human-simulated intelligent control (HSIC). This HF algorithm incorporates elements of both multi-mode proportional-derivative (PD) and fuzzy control, aiming at solving the chattering problem of multi-mode switching while improving control accuracy. The second is a membrane architecture that makes use of the natural parallelisation potential of membrane computing to improve the real-time performance of the controller. The proposed HFMC is utilised as the joint controller for a biped robot. Numerical tests in a simulation are carried out with the planar and slope walking of a five-link biped robot, and the effectiveness of the HFMC is verified by comparing and evaluating the results of the designed HFMC, HSIC and PD. Experimental results demonstrate that the proposed HFMC not only retains the advantages of traditional PD control but also improves control accuracy, real-time performance and stability.

show abstract

Section: Introductionmentioning

confidence: 99%

A human-simulated fuzzy membrane approach for the joint controller of walking biped robots

Liu

Zhang

Mastoi

et al. 2023

ICA

View full text Add to dashboard Cite

show abstract

“…The proposed approach used a multi-layered critic to identify a unique value function for each reward component, resulting in hybrid policy gradients. To construct a feedback system that can handle the walking pattern problem, a feedback system that combines an Adaptive Neural Fuzzy Inference System (ANFIS) [51] and a Double Deep Q-network (DDQN) [52] was proposed in [53]. To update the walking parameters, the output of the ANFIS was utilized for training a predictive model called DDQN.…”

Section: Introductionmentioning

confidence: 99%

Development of a New Robust Stable Walking Algorithm for a Humanoid Robot Using Deep Reinforcement Learning with Multi-Sensor Data Fusion

2023

View full text Add to dashboard Cite

The difficult task of creating reliable mobility for humanoid robots has been studied for decades. Even though several different walking strategies have been put forth and walking performance has substantially increased, stability still needs to catch up to expectations. Applications for Reinforcement Learning (RL) techniques are constrained by low convergence and ineffective training. This paper develops a new robust and efficient framework based on the Robotis-OP2 humanoid robot combined with a typical trajectory-generating controller and Deep Reinforcement Learning (DRL) to overcome these limitations. This framework consists of optimizing the walking trajectory parameters and posture balancing system. Multi-sensors of the robot are used for parameter optimization. Walking parameters are optimized using the Dueling Double Deep Q Network (D3QN), one of the DRL algorithms, in the Webots simulator. The hip strategy is adopted for the posture balancing system. Experimental studies are carried out in both simulation and real environments with the proposed framework and Robotis-OP2’s walking algorithm. Experimental results show that the robot performs more stable walking with the proposed framework than Robotis-OP2’s walking algorithm. It is thought that the proposed framework will be beneficial for researchers studying in the field of humanoid robot locomotion.

show abstract

“…30 Moreover, DRL algorithms have been implemented to accomplish diverse engineering tasks. For example, it is challenging to use robots in the real world, but DRL algorithms can provide model-free solutions for operating humanoid robots 31 and wheeled robots. 32 Chaotic systems are highly similar to such robots, in that they have relatively complex and unpredictable characteristics.…”

Section: Introductionmentioning

confidence: 99%

Proximal policy optimization‐based controller for chaotic systems

Yau,

Kuo,

Luan

et al. 2023

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

Deep reinforcement learning (DRL) algorithms are suitable for modeling and controlling complex systems. Methods for controlling chaos, a difficult task, require improvement. In this article, we present a DRL‐based control method that can control a nonlinear chaotic system without any prior knowledge of the system's equations. We use proximal policy optimization (PPO) to train an agent. The environment is a Lorenz chaotic system, and our goal is to stabilize this chaotic system as quickly as possible and minimize the error by adding extra control terms to the chaotic system. Therefore, the reward function accounts for the total triaxial error. The experimental results demonstrated that the trained agent can rapidly suppress chaos in the system, regardless of the system's random initial conditions. A comprehensive comparison of different DRL algorithms indicated that PPO is the most efficient and effective algorithm for controlling the chaotic system. Moreover, different maximum control forces were applied to determine the relationship between the control forces and controller performance. To verify the robustness of the controller, random disturbances were introduced during training and testing, and the empirical results indicated that the agent trained with random noise performed better. The chaotic system has highly nonlinear characteristics and is extremely sensitive to initial conditions, and DRL is suitable for modeling such systems.

show abstract

Fuzzy Double Deep Q-Network-Based Gait Pattern Controller for Humanoid Robots

Cited by 13 publications

References 37 publications

A human-simulated fuzzy membrane approach for the joint controller of walking biped robots

A human-simulated fuzzy membrane approach for the joint controller of walking biped robots

Development of a New Robust Stable Walking Algorithm for a Humanoid Robot Using Deep Reinforcement Learning with Multi-Sensor Data Fusion

Proximal policy optimization‐based controller for chaotic systems

Contact Info

Product

Resources

About