Optimizing the agent decisions for a Cloud actuator using Deep reinforcement learning

Abstract

With the increasing use of deep reinforcement learning (DRL) techniques to build intelligent systems, the application of it to real-world problems is rampant. Resource allocation in a cloud environment that need dynamic and auto-scaling features is evolving. The agent-based decisions that are offered by DRL are in use by software robotics. Auto-scaling of resources in cloud applications introduces intelligence to agents thus built by these DRL techniques. Markov decision process as a tool minimizes the target rewards to agents such that auto-scaling of applications is performed by agent decisions. Analysis of optimizing the convergence errors that are measured while the agent performs in an online environment is the challenge. Speedy Q-learning (SQL), Generalized SQL(GSQL) algorithm variants relax the parameter values of convergence with a model-free space. The authors applied heuristic values for one such relaxation parameter in our experiments. The study is an extension of works that introduced GSQL-w, where w is the convergence parameter. The authors designed a new GSQL-wh algorithm that heuristically fixes a value for w optimally in cases with over-utilization of resources. This is presented as a novel solution in this study for cloud resource workloads.