Deep Reinforcement Learning with Successive Over-Relaxation and its Application in Autoscaling Cloud Resources

John, Indu; Bhatnagar, Shalabh

doi:10.1109/ijcnn48605.2020.9206598

Cited by 4 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the GSQL-w algorithm, the w-parameter [3] is for convergence of errors, de ning a few options to consider, depending on the possible choice of its value. In a system that needs to realize the optimization of resources, the selection of w determines two cases.…”

Section: Methodsmentioning

confidence: 99%

“…Our motivation is hence to perform the experiments by extending the GSQL-w for varying w, parallel also for several space and action values. This is our novel attempt at heuristics on w, the convergence parameter that was a limitation in [3]. The steps involved in our new framework are presented in the subsection.…”

Section: Methodsmentioning

confidence: 99%

“…However, not many of the surveyed works propose a solution that covers either scaling or schedule, but not both [1]. Applying the modi ed SQL algorithms demonstrates that the agents learn faster from the environment in the online mode thereby allowing the over-utilization parameters that could be optimized [3][4] [5]. That is proof that auto scalars with reinforcement bring agents closer to intelligence systems.…”

Section: Problem Statementmentioning

confidence: 99%

See 2 more Smart Citations

Optimizing the agent decisions for a Cloud actuator using Deep reinforcement learning

Sankaran,

JS,

Suleiman

2024

Preprint

View full text Add to dashboard Cite

With the increasing use of deep reinforcement learning (DRL) techniques to build intelligent systems, the application of it to real-world problems is rampant. Resource allocation in a cloud environment that need dynamic and auto-scaling features is evolving. The agent-based decisions that are offered by DRL are in use by software robotics. Auto-scaling of resources in cloud applications introduces intelligence to agents thus built by these DRL techniques. Markov decision process as a tool minimizes the target rewards to agents such that auto-scaling of applications is performed by agent decisions. Analysis of optimizing the convergence errors that are measured while the agent performs in an online environment is the challenge. Speedy Q-learning (SQL), Generalized SQL(GSQL) algorithm variants relax the parameter values of convergence with a model-free space. The authors applied heuristic values for one such relaxation parameter in our experiments. The study is an extension of works that introduced GSQL-w, where w is the convergence parameter. The authors designed a new GSQL-wh algorithm that heuristically fixes a value for w optimally in cases with over-utilization of resources. This is presented as a novel solution in this study for cloud resource workloads.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Problem Statementmentioning

confidence: 99%

See 1 more Smart Citation

Optimizing the agent decisions for a Cloud actuator using Deep reinforcement learning

Sankaran,

JS,

Suleiman

2024

Preprint

View full text Add to dashboard Cite

show abstract

Automated cloud resources provisioning with the use of the proximal policy optimization

2022

View full text Add to dashboard Cite

Many modern applications, both scientific and commercial, are deployed to cloud environments and often employ multiple types of resources. That allows them to efficiently allocate only the resources which are actually needed to achieve their goals. However, in many workloads the actual usage of the infrastructure varies over time, which results in over-provisioning and unnecessarily high costs. In such cases, automatic resource scaling can provide significant cost savings by provisioning only the amount of resources which are necessary to support the current workload. Unfortunately, due to the complex nature of distributed systems, automatic scaling remains a challenge. Reinforcement learning domain has been recently a very active field of research. Thanks to combining it with Deep Learning, many newly designed algorithms improve the state of the art in many complex domains. In this paper we present the results of our attempt to use the recent advancements in Reinforcement Learning to optimize the cost of running a compute-intensive evolutionary process by automating the scaling of heterogeneous resources in a compute cloud environment. We describe the architecture of our system and present evaluation results. The experiments include autonomous management of a sample workload and a comparison of its performance to the traditional automatic threshold-based management approach. We also provide the details of training of the management policy using the proximal policy optimization algorithm. Finally, we discuss the feasibility to extend the presented approach to further scenarios.

show abstract

Learning-driven hybrid scaling for multi-type services in cloud

Zhang,

Guo,

Tian

et al. 2024

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

Deep Reinforcement Learning with Successive Over-Relaxation and its Application in Autoscaling Cloud Resources

Cited by 4 publications

References 13 publications

Optimizing the agent decisions for a Cloud actuator using Deep reinforcement learning

Optimizing the agent decisions for a Cloud actuator using Deep reinforcement learning

Automated cloud resources provisioning with the use of the proximal policy optimization

Learning-driven hybrid scaling for multi-type services in cloud

Contact Info

Product

Resources

About