TeXDYNA: Hierarchical Reinforcement Learning in Factored MDPs

Kozlova, Olga; Sigaud, Olivier; Meyer, Christophe

doi:10.1007/978-3-642-15193-4_46

Cited by 2 publications

(1 citation statement)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We give an overview of existing factored Reinforcement Learning approaches. Most of the works consider graphical representations of trees [14] or a DBN [6]. These two objects allow to represent relations between state variables and conditional probabilities in a compact form.…”

Section: Factored Reinforcement Learningmentioning

confidence: 99%

Factored Reinforcement Learning for Auto-scaling in Tandem Queues

Tournaire

Jin

Aghasaryan

et al. 2022

NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium

View full text Add to dashboard Cite

As today's networking systems utilises more virtualisation, efficient auto-scaling of resources becomes increasingly critical for controlling both the performance and energy consumption. In this paper, we study the techniques to learn the optimal auto-scaling policies in a distributed network when parts of the system dynamics are unknown. Reinforcement Learning methods have been applied to solve auto-scaling problems. However they can run into computational and convergence issues as the problem scale grows. On the other hand, distributed networks have relational structures with local dependencies between physical and virtual resources. We can exploit these structures to overcome the convergence issues by using a factored representation of the system.We consider a distributed network in the form of a tandem queue composed of two nodes. The objective of the autoscaling problem is to find policies that have a good trade-off between quality of service (QoS) and operating costs. We develop a factored Reinforcement Learning algorithm, named FMDP online, to find the optimal auto-scaling policies. We evaluate our algorithm with a simulated environment. We compare it with existing Reinforcement Learning methods and show its relevance in terms of policy efficiency and convergence speed.

show abstract

Section: Factored Reinforcement Learningmentioning

confidence: 99%