Scalable transfer learning in heterogeneous, dynamic environments

Abstract

Reinforcement learning is a plausible theoretical basis for developing self-learning, autonomous agents or robots that can effectively represent the world dynamics and efficiently learn the problem features to perform different tasks in different environments. The computational costs and complexities involved, however, are often prohibitive for real-world applications. This study introduces a scalable methodology to learn and transfer knowledge of the transition (and reward) models for model-based reinforcement learning in a complex world. We propose a variant formulation of Markov decision processes that supports efficient online-learning of the relevant problem features to approximate the world dynamics. We apply the new feature selection and dynamics approximation techniques in heterogeneous transfer learning, where the agent automatically maintains and adapts multiple representations of the world to cope with the different environments it encounters during its lifetime. We prove regret bounds for our approach, and empirically demonstrate its capability to quickly converge to a near optimal policy in both real and simulated environments.

Towards a science of integrated AI and Robotics

“…Papers [13] and [16] also deal with learning, but put a strong emphasis on managing the complexity of doing so, thus addressing RC3. [13] introduces a scalable methodology to learn and transfer knowledge of the transition (and reward) models for model-based reinforcement learning in a complex world.…”

“…[13] introduces a scalable methodology to learn and transfer knowledge of the transition (and reward) models for model-based reinforcement learning in a complex world. The authors use a formulation of Markov decision processes that support efficient online-learning of relevant problem features in order to approximate world dynamics.…”

Local Energy Management and Optimization: A Novel Energy Universal Service Bus System Based on Energy Internet Technologies

“…It combines the improved TOPSIS (technique for order preference similar to an ideal solution) algorithm in terms of multi-objective decision and the multi-step backtracking Q(λ) algorithm about random optimization ability, so that it is remarkably applied in solving of real-time dynamic control issues of active loads. (c) TRL [66][67][68][69]. It is a novel algorithm based on a high integration of multi-agent collaboration, reinforcement learning and transfer learning in term of an efficient information utilization of historical optimization tasks, perceptibly can be applied in field of fast dynamic optimization of active loads.…”

Unsupervised Anomaly Detection Using Intelligent and Heterogeneous Autonomous Systems: Highlights From the 2020 IEEE Signal Processing Cup Student Competition [SP Competitions]

“…The nature of knowledge should be ex plainable and transferable to other agents. In [3], the authors point out that there are still many open problems and assumptions to address, where a transfer learning framework would serve as an intelligent base platform for monitoring, problem solving, and general decision support in heterogeneous dynamic environments, leading to a new generation of human-interactive robots.…”