A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots

Phaniteja, S; Dewangan, Parijat; Guhan, Pooja; Sarkar, Abhishek; Krishna, K. Madhava

doi:10.1109/robio.2017.8324682

Cited by 47 publications

(23 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…the average errors from 50 individual trails with respect to learning episodes while applying the proposed HRL and applying HGP as the pure model-based RL, respectively. The error is defined as the Euler distance from the desired CoP to the measured CoP, which can be calculated using (13). The error at each episode is computed by averaging the values from 40 randomly initialized trails at the same episode.…”

Section: B Experiments Resultsmentioning

confidence: 99%

“…The simulation results showed that the proposed algorithm was able to efficiently learn humanoid motor skills to complete tasks. In 2017, Phaniteja [13], employed Deep Deterministic Policy Gradient (DDPG) to learn a robust Inverse Kinematic (IK) solver and obtain stable IK solutions. The robot was able to complete the tasks involving reaching a point in the far right, in the left-back side, and blew its knee, respectively.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Walking Control of a Biped Robot on Static and Rotating Platforms Based on Hybrid Reinforcement Learning

Chen

2020

IEEE Access

View full text Add to dashboard Cite

In this paper, we proposed a novel Hybrid Reinforcement Learning framework to maintain the stability of a biped robot (NAO) while it is walking on static and dynamic platforms. The reinforcement learning framework consists of the Model-based off-line Estimator, the Actor Network Pre-training scheme, and the Mode-free on-line optimizer. We proposed the Hierarchical Gaussian Processes as the Mode-based Estimator to predict a rough model of the system and to obtain the initial control input. Then, the initial control input is employed to pre-train the Actor Network by using the initial control input. Finally, a modelfree optimizer based on Deep Deterministic Policy Gradient framework is introduced to fine tune the Actor Network and to generate the best actions. The proposed reinforcement learning framework not only successfully avoids the distribution mismatch problem while combining model-based scheme with modelfree structure, but also improves the sample efficiency for the on-line learning procedure. Simulation results show that the proposed Hybrid Reinforcement Learning mechanism enables the NAO robot to maintain balance while walking on static and dynamic platforms. The robustness of the learned controllers in adapting to platforms with different angles, different magnitudes, and different frequencies is tested.

show abstract

Section: B Experiments Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Walking Control of a Biped Robot on Static and Rotating Platforms Based on Hybrid Reinforcement Learning

Chen

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Against this background, a number of researchers have adopted deep reinforcement learning (DRL) to achieve breakthroughs in automatic control [14] and game competition [15,16] and better solve cloud computing scheduling problems. Liu et al [17] devised a DRL-based hierarchical system resource allocation and power management framework, which includes a global layer to allocate VMs to physical servers and a local layer for server power management.…”

Section: Introductionmentioning

confidence: 99%

A Two-Stage Framework for the Multi-User Multi-Data Center Job Scheduling and Resource Allocation

et al. 2020

View full text Add to dashboard Cite

With the development of artificial intelligence and the Internet of things, the prospects of cloud computing applications have become broader, and the number of users and cloud data centers (CDCs) has exploded. It is a challenge to realize efficient job scheduling and resource allocation of multiple users and data centers. However, the traditional scheduling model based on heuristic algorithm has some limitations in the complex and changeable cloud environment. In addition, many existing single-agent models rarely consider the multi-objective global optimization of the system. Therefore, this paper proposes a two-stage job scheduling and resource allocation framework that adopts multiple intelligent schedulers to solve the cooperative scheduling problem between job scheduling and resource allocation. A heterogeneous distributed deep learning (HDDL) model is used in the job scheduling stage to schedule multiple jobs to multiple cloud data centers. The deep Q-network (DQN) model is a resource scheduler to deploy virtual machine to physical servers for execution. Extensive numerical results show that both HDDL-baesd job scheduler and DQN-based resource allocator outperform the benchmark algorithm in terms of optimizing energy consumption and job delay. Furthermore, the proposed framework not only can achieve a global near-optimum by achieving local optimization at each stage but also has good scalability and low computation delay.

show abstract

“…Deep reinforcement learning has been applied in many aspects, e.g., natural language process [17], gaming [18], and robot control [19]. It uses a deep neural network (DNN) to empirically solve large-scale complex problems.…”

Section: Introductionmentioning

confidence: 99%

Multi-Server Multi-User Multi-Task Computation Offloading for Mobile Edge Computing Networks

Huang

Zhang

et al. 2019

Sensors

View full text Add to dashboard Cite

This paper studies mobile edge computing (MEC) networks where multiple wireless devices (WDs) offload their computation tasks to multiple edge servers and one cloud server. Considering different real-time computation tasks at different WDs, every task is decided to be processed locally at its WD or to be offloaded to and processed at one of the edge servers or the cloud server. In this paper, we investigate low-complexity computation offloading policies to guarantee quality of service of the MEC network and to minimize WDs’ energy consumption. Specifically, both a linear programing relaxation-based (LR-based) algorithm and a distributed deep learning-based offloading (DDLO) algorithm are independently studied for MEC networks. We further propose a heterogeneous DDLO to achieve better convergence performance than DDLO. Extensive numerical results show that the DDLO algorithms guarantee better performance than the LR-based algorithm. Furthermore, the DDLO algorithm generates an offloading decision in less than 1 millisecond, which is several orders faster than the LR-based algorithm.

show abstract

A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots

Cited by 47 publications

References 17 publications

Walking Control of a Biped Robot on Static and Rotating Platforms Based on Hybrid Reinforcement Learning

Walking Control of a Biped Robot on Static and Rotating Platforms Based on Hybrid Reinforcement Learning

A Two-Stage Framework for the Multi-User Multi-Data Center Job Scheduling and Resource Allocation

Multi-Server Multi-User Multi-Task Computation Offloading for Mobile Edge Computing Networks

Contact Info

Product

Resources

About