DreamWaQ: Learning Robust Quadrupedal Locomotion With Implicit Terrain Imagination via Deep Reinforcement Learning

Nahrendra, I Made Aswin; Yu, Byeongho; Myung, Hyun

doi:10.1109/icra48891.2023.10161144

Cited by 21 publications

(10 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each task in the sequence is a robot locomotion task, which we model as an RL task. According to [ 19 ], we define a Q-function

, a policy

and a context-aided estimator network (CENet)

. Q-learning methods train a Q-function by iteratively applying the Bellman operator

.…”

Section: Resultsmentioning

confidence: 99%

“…By introducing the privileged observation

only into the Q-network, the agent (policy) also can makes decisions without privileged observation when evaluating in the real world. The implementation also follows the definition of the privileged observation

in [ 19 ]. The CENet from [ 19 ] is used to jointly learn to estimate and infer a latent representation of the environment.…”

Section: Resultsmentioning

confidence: 99%

“…The implementation also follows the definition of the privileged observation

in [ 19 ]. The CENet from [ 19 ] is used to jointly learn to estimate and infer a latent representation of the environment. The architecture of the CENet consists of a single encoder and a multi-head decoder to encode

into

and

.…”

Section: Resultsmentioning

confidence: 99%

“…Our basic actor-critical RL method is from [ 19 ]. For details on the specific state space

, action space

, and reward function r , we follow [ 19 ].…”

Section: Resultsmentioning

confidence: 99%

“…Our basic actor-critical RL method is from [ 19 ]. For details on the specific state space

, action space

, and reward function r , we follow [ 19 ]. We also list the elements of the reward function in Table 1 , where

indicates the desired values.…”

Section: Resultsmentioning

confidence: 99%

See 4 more Smart Citations

Continual Reinforcement Learning for Quadruped Robot Locomotion

Gai,

Lyu,

Zhang

et al. 2024

Entropy

View full text Add to dashboard Cite

The ability to learn continuously is crucial for a robot to achieve a high level of intelligence and autonomy. In this paper, we consider continual reinforcement learning (RL) for quadruped robots, which includes the ability to continuously learn sub-sequential tasks (plasticity) and maintain performance on previous tasks (stability). The policy obtained by the proposed method enables robots to learn multiple tasks sequentially, while overcoming both catastrophic forgetting and loss of plasticity. At the same time, it achieves the above goals with as little modification to the original RL learning process as possible. The proposed method uses the Piggyback algorithm to select protected parameters for each task, and reinitializes the unused parameters to increase plasticity. Meanwhile, we encourage the policy network exploring by encouraging the entropy of the soft network of the policy network. Our experiments show that traditional continual learning algorithms cannot perform well on robot locomotion problems, and our algorithm is more stable and less disruptive to the RL training progress. Several robot locomotion experiments validate the effectiveness of our method.

show abstract

“…Each task in the sequence is a robot locomotion task, which we model as an RL task. According to [ 19 ], we define a Q-function

, a policy

and a context-aided estimator network (CENet)

. Q-learning methods train a Q-function by iteratively applying the Bellman operator

.…”

Section: Resultsmentioning

confidence: 99%

“…By introducing the privileged observation

in [ 19 ]. The CENet from [ 19 ] is used to jointly learn to estimate and infer a latent representation of the environment.…”

Section: Resultsmentioning

confidence: 99%