Auto-scale: automatic scaling of virtualised resources using neuro-fuzzy reinforcement learning approach

Veni, T.; Bhanu, S. Mary Saira

doi:10.1504/ijbdi.2016.078400

Cited by 18 publications

(15 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, it is not suitable under dynamic workloads as states are formulated as a function of workload and capacity allocation. The closest technique to the proposed mechanism is Auto-scale [24], an RL-based technique for adaptive QoS control via vertical scaling using Neuro-Fuzzy function approximations. However, the focus is only on meeting performance target without regards for efficiency.…”

Section: Related Workmentioning

confidence: 99%

Adaptive Service Performance Control using Cooperative Fuzzy Reinforcement Learning in Virtualized Environments

Ibidunmoye

Moghadam

Lakew

et al. 2017

Proceedings of The10th International Conference on Utility and Cloud Computing

View full text Add to dashboard Cite

Designing efficient control mechanisms to meet strict performance requirements with respect to changing workload demands without sacrificing resource efficiency remains a challenge in cloud infrastructures. A popular approach is fine-grained resource provisioning via auto-scaling mechanisms that rely on either threshold-based adaptation rules or sophisticated queuing/control-theoretic models. While it is difficult at design time to specify optimal threshold rules, it is even more challenging inferring precise performance models for the multitude of services. Recently, reinforcement learning have been applied to address this challenge. However, such approaches require many learning trials to stabilize at the beginning and when operational conditions vary thereby limiting their application under dynamic workloads. To this end, we extend the standard reinforcement learning approach in two ways: a) we formulate the system state as a fuzzy space and b) exploit a set of cooperative agents to explore multiple fuzzy states in parallel to speed up learning. Through multiple experiments on a real virtualized testbed, we demonstrate that our approach converges quickly, meets performance targets at high efficiency without explicit service models.

show abstract

Section: Related Workmentioning

confidence: 99%

Adaptive Service Performance Control using Cooperative Fuzzy Reinforcement Learning in Virtualized Environments

Ibidunmoye

Moghadam

Lakew

et al. 2017

Proceedings of The10th International Conference on Utility and Cloud Computing

View full text Add to dashboard Cite

show abstract

“…Machine learning has been widely used for the analysis of data resulting from performance testing and also for performance preservation. For example, anomaly detection through analysis of performance data, e.g., resource usage, using clustering techniques (Syer et al 2011), predicting reliability from the testing data using Bayesian Networks (Avritzer et al 2008), performance signature identification based on performance data analysis using supervised and unsupervised learning techniques (Malik et al 2013;Malik et al 2010), and also adaptive RL-driven performance in particular response time control for cloud services (Ibidunmoye et al 2017;Veni and Bhanu 2016;Jamshidi et al 2016) and also software on other execution platforms, e.g., PLCbased real-time systems (Moghadam et al 2018). Machine learning has been also applied to the generation of performance test cases in some studies.…”

Section: Related Workmentioning

confidence: 99%

An autonomous performance testing framework using self-adaptive fuzzy reinforcement learning

et al. 2021

View full text Add to dashboard Cite

Test automation brings the potential to reduce costs and human effort, but several aspects of software testing remain challenging to automate. One such example is automated performance testing to find performance breaking points. Current approaches to tackle automated generation of performance test cases mainly involve using source code or system model analysis or use-case-based techniques. However, source code and system models might not always be available at testing time. On the other hand, if the optimal performance testing policy for the intended objective in a testing process instead could be learned by the testing system, then test automation without advanced performance models could be possible. Furthermore, the learned policy could later be reused for similar software systems under test, thus leading to higher test efficiency. We propose SaFReL, a self-adaptive fuzzy reinforcement learning-based performance testing framework. SaFReL learns the optimal policy to generate performance test cases through an initial learning phase, then reuses it during a transfer learning phase, while keeping the learning running and updating the policy in the long term. Through multiple experiments in a simulated performance testing setup, we demonstrate that our approach generates the target performance test cases for different programs more efficiently than a typical testing process and performs adaptively without access to source code and performance models.

show abstract

“…Thus, although most of the proposed solutions for the autoscaling problem of workflows in Cloud are based on heuristics, several recent research aims to apply reinforcement learning approaches to solve any of the subproblems involved, the scaling [10,11,12,13,14,15,16,8] or scheduling [17,18,19,20,21,22,23], which are characterized in Cloud by the conditions of uncertainty.…”

Section: Scalingmentioning

confidence: 99%

“…The uncertainty in these problems is associated with the variability in the performance of the Cloud infrastructure. In this sense, proposals that model the autoscaling problem as an MDP and use different RL techniques to learn adequate scaling policies [10,11,12,13,14,15,16,8] or scheduling [17,18,19,20,21,22,23] have appeared. These policies allow an autoscaler to determine which action is more convenient at any time, in order to optimize in the long-term one or more objectives from the point of view of the execution of one, or a set of applications/workflows.…”

Section: Review Of Cloud Autoscaling Based On Rl Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement Learning-based Application Autoscaling in the Cloud: A Survey

Garí¹,

Monge²,

Pacini³

et al. 2020

Preprint

View full text Add to dashboard Cite

Reinforcement Learning (RL) has demonstrated a great potential for automatically solving decision making problems in complex uncertain environments. Basically, RL proposes a computational approach that allows learning through interaction in an environment of stochastic behavior, with agents taking actions to maximize some cumulative short-term and long-term rewards. Some of the most impressive results have been shown in Game Theory where agents exhibited super-human performance in games like Go or Starcraft 2, which led to its adoption in many other domains including Cloud Computing. Particularly, workflow autoscaling exploits the Cloud elasticity to optimize the execution of workflows according to a given optimization criteria. This is a decision-making problem in which it is necessary to establish when and how to scale-up/down computational resources; and how to assign them to the upcoming processing workload. Such actions have to be taken considering some optimization criteria in the Cloud, a dynamic and uncertain environment. Motivated by this, many works apply RL to the autoscaling problem in Cloud. In this work we survey exhaustively those proposals from major venues, and uniformly compare them based on a set of proposed taxonomies. We also discuss open problems and provide a prospective of future research in the area.

show abstract

Auto-scale: automatic scaling of virtualised resources using neuro-fuzzy reinforcement learning approach

Cited by 18 publications

References 0 publications

Adaptive Service Performance Control using Cooperative Fuzzy Reinforcement Learning in Virtualized Environments

Adaptive Service Performance Control using Cooperative Fuzzy Reinforcement Learning in Virtualized Environments

An autonomous performance testing framework using self-adaptive fuzzy reinforcement learning

Reinforcement Learning-based Application Autoscaling in the Cloud: A Survey

Contact Info

Product

Resources

About