Reinforcement Learning Techniques for Decentralized Self-adaptive Service Assembly

A self-adaptive system can automatically maintain its quality requirements in the presence of dynamic environment changes. Developing a self-adaptive system may be difficult due to design time uncertainty; e.g., anticipating all potential environment changes at design time is in most cases infeasible. To realize self-adaptive systems in the presence of design time uncertainty, online machine learning, i.e., machine learning at runtime, is increasingly used. In particular, online reinforcement learning is proposed, which learns suitable adaptation actions through interactions with the environment at runtime. To learn about its environment, online reinforcement learning has to select actions that were not selected before, which is known as exploration. How exploration happens impacts the performance of the learning process. We focus on two problems related to how adaptation actions are explored. First, existing solutions randomly explore adaptation actions and thus may exhibit slow learning if there are many possible adaptation actions. Second, they are unaware of system evolution, and thus may explore new adaptation actions introduced during evolution rather late. We propose novel exploration strategies that use feature models (from software product line engineering) to guide exploration in the presence of many adaptation actions and system evolution. Experimental results for two realistic self-adaptive systems indicate an average speed-up of the learning process of 33.7% in the presence of many adaptation actions, and of 50.6% in the presence of evolution.

show abstract

“…To facilitate reproducibility and replicability, our code, the used data and our experimental results are available online. 7…”

Section: Resultsmentioning

confidence: 99%

Realizing self-adaptive systems via online reinforcement learning and feature-model-guided exploration

et al. 2022

View full text Add to dashboard Cite

show abstract

“…IETF's Autonomic Networking Integrated Model and Approach (ANIMA) [28] working group explores autonomic networking of managed devices with the goal to provide the self-x properties, however, control loops are currently not covered. Some proposals include learning ( [22], [29]- [33]), for example [22] uses agents to implement cognitive control loops, and [33] proposes a multi-agent network automation architecture that follows the Generic Autonomic Network Architecture (GANA) reference model from the European Telecommunication Standards Institution (ETSI) [34]. The failure to deploy autonomic network management in the past is discussed and reconsidered in light of new enabler technologies in [24].…”

Section: B Autonomic Network Managementmentioning

confidence: 99%

“…When 7−2 is enforced in line 19, the monitorData method (line 23) is called. This method uses the populated link utilization category and iterates over each pair of resource and metric to collect the monitoring data (lines [25][26][27][28][29][30][31]. In use-case, each resource is a link with a metric type that is either core or edge, hence the output is a key-value data structure separated by metric type, where each key is a link name and the value is the link utilization.…”

Section: Use-casementioning

confidence: 99%

Towards a Self-Driving Management System for the Automated Realization of Intents

Dzeparoska¹,

Beigi-Mohammadi

Tizghadam

et al. 2021

IEEE Access

View full text Add to dashboard Cite

“…To realize the service choreography, each peer needs to discover the services offered by other peers, together with their utilization level, so to determine the best mapping that satisfy the workflow requirements (e.g., minimum response time, maximum throughput). Similarly to the approach presented in [15], the system can employ the information sharing pattern to share, among peers, knowledge about the services offered by peers and their utilization state. Relying on this information, at run-time, the service choreography can be adapted so to automatically scale the number of concrete services to be used to run the workflow.…”

Section: Patternmentioning

confidence: 99%

QoS-Based Elasticity for Service Chains in Distributed Edge Cloud Environments

Cardellini

Grbac

Nardelli

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

With the emerging IoT and Cloud-based networked systems that rely heavily on virtualization technologies, elasticity becomes a dominant system engineering attribute for providing QoS-aware services to their users. Although the concept of elasticity can introduce significant QoS and cost benefits, its implementation in real systems is full of challenges. Indeed, nowadays systems are mainly distributed, built upon several layers of abstraction, and with centralized control mechanisms. In such a complex environment, controlling elasticity in a centralized manner might strongly penalize scalability. To overcome this issue, we can conveniently split the system in autonomous subsystems that implement elasticity mechanisms and run control policies in a decentralized manner. To efficiently and effectively cooperate with each other, the subsystems need to communicate among themselves to determine elasticity decisions that collectively improve the overall system performance. This new architecture calls for the development of new mechanisms and efficient policies. In this chapter, we focus on elasticity in IoT and Cloud-based systems, which can be geo-distributed also at the edge of the networks, and discuss its engineering perspectives along with various coordination mechanisms. We focus on the design choices that may affect the elasticity properties and provide an overview of some decentralized design patterns related to the coordination of elasticity decisions.

show abstract

Reinforcement Learning Techniques for Decentralized Self-adaptive Service Assembly

Cited by 13 publications

References 17 publications

Realizing self-adaptive systems via online reinforcement learning and feature-model-guided exploration

Realizing self-adaptive systems via online reinforcement learning and feature-model-guided exploration

Towards a Self-Driving Management System for the Automated Realization of Intents

QoS-Based Elasticity for Service Chains in Distributed Edge Cloud Environments

Contact Info

Product

Resources

About