Independent Skill Transfer for Deep Reinforcement Learning

Abstract

<jats:p>Recently, diverse primitive skills have been learned by adopting the entropy as intrinsic reward, which further shows that new practical skills can be produced by combining a variety of primitive skills. This is essentially skill transfer, very useful for learning high-level skills but quite challenging due to the low efficiency of transferring primitive skills. In this paper, we propose a novel efficient skill transfer method, where we learn independent skills and only independent components of skills are transferred instead of the whole set of skills. More concretely, independent components of skills are obtained through independent component analysis (ICA), which always have a smaller amount (or lower dimension) compared with their mixtures. With a lower dimension, independent skill transfer (IST) exhibits a higher efficiency on learning a given task. Extensive experiments including three robotic tasks demonstrate the effectiveness and high efficiency of our proposed IST method in comparison to direct primitive-skill transfer and conventional reinforcement learning.</jats:p>

Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learning

“…Thus, the desirability of our approach is that the acquired reward function uncovers both the source dynamics (q φ ) and the dynamics difference (β∆r) across source and target environment. Complementary to our work, several other works also encourage the emergence of a state-covering goal distribution [37,6,27] or enable transfer by introducing the regularization over policies [45,15,46,47,36,48] instead of the adaptation over different dynamics.…”

Exploration and Exploitation in Hierarchical Reinforcement Learning with Adaptive Scheduling

“…DIAYN [7] created the inferred relationship between options, states, and actions. IST [30] encourages the agent to discover states that are challenging to the policy so that more complex actions can be obtained through options. HIDIO [33] employed continuous options for the first time and compared the effects of different mutual information forms.…”

“…The limited * Corresponding Author. Email:quanliu@suda.edu.cn number requires them to be empirically explainable [28] and nonredundant [30]. Even as the number of discrete options increases, it is still unrealistic for the option policy to traverse each option value.…”