Online Reinforcement Learning for Self-adaptive Information Systems

Palm, Alexander; Metzger, Andreas; Pohl, Klaus

doi:10.1007/978-3-030-49435-3_11

Cited by 36 publications

(23 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2, data-driven algorithms could be applied to enhance the MAPE-K with automatic generation of policies. The integration of ML and reinforced learning into MAPE-K model as suggested in [49] should be taken into consideration for this direction.…”

Section: Discussionmentioning

confidence: 99%

Dynamical Orchestration and Configuration Services in Industrial IoT Systems: An Autonomic Approach

Lam

Haugen

Delsing

2022

IEEE Open J. Ind. Electron. Soc.

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

Dynamical Orchestration and Configuration Services in Industrial IoT Systems: An Autonomic Approach

Lam

Haugen

Delsing

2022

IEEE Open J. Ind. Electron. Soc.

View full text Add to dashboard Cite

show abstract

“…For a self-adaptive system, "agent" refers to the self-adaptation logic of the system and "action" refers to an adaptation action [30]. In the integrated model, action selection of reinforcement learning takes the place of the analyze and plan activities of MAPE-K.…”

Section: Reinforcement Learning and Self-adaptationmentioning

confidence: 99%

Realizing self-adaptive systems via online reinforcement learning and feature-model-guided exploration

et al. 2022

Self Cite

View full text Add to dashboard Cite

A self-adaptive system can automatically maintain its quality requirements in the presence of dynamic environment changes. Developing a self-adaptive system may be difficult due to design time uncertainty; e.g., anticipating all potential environment changes at design time is in most cases infeasible. To realize self-adaptive systems in the presence of design time uncertainty, online machine learning, i.e., machine learning at runtime, is increasingly used. In particular, online reinforcement learning is proposed, which learns suitable adaptation actions through interactions with the environment at runtime. To learn about its environment, online reinforcement learning has to select actions that were not selected before, which is known as exploration. How exploration happens impacts the performance of the learning process. We focus on two problems related to how adaptation actions are explored. First, existing solutions randomly explore adaptation actions and thus may exhibit slow learning if there are many possible adaptation actions. Second, they are unaware of system evolution, and thus may explore new adaptation actions introduced during evolution rather late. We propose novel exploration strategies that use feature models (from software product line engineering) to guide exploration in the presence of many adaptation actions and system evolution. Experimental results for two realistic self-adaptive systems indicate an average speed-up of the learning process of 33.7% in the presence of many adaptation actions, and of 50.6% in the presence of evolution.

show abstract

“…They have been proven to work well even without extensive hyper-parametrisation, provided that enough good-quality data is available. This means that the time-and resource-consuming step of extensive experimentation with hyper-parameters may be skipped, leading to a more efficient development and deployment process of big data applications (Palm et al 2020). • Data accuracy: Operators benefit from information about data accuracy.…”

Section: Data Analyticsmentioning

confidence: 99%

A Reference Model for Big Data Technologies

Curry

Metzger

Berre

et al. 2021

The Elements of Big Data Value

Self Cite

View full text Add to dashboard Cite

The Big Data Value (BDV) Reference Model has been developed with input from technical experts and stakeholders along the whole big data value chain. The BDV Reference Model may serve as a common reference framework to locate big data technologies on the overall IT stack. It addresses the main technical concerns and aspects to be considered for big data value systems. The BDV Reference Model enables the mapping of existing and future data technologies within a common framework. Within this chapter, we detail the reference model in more detail and show how it can be used to manage a portfolio of research and innovation projects.

show abstract

Online Reinforcement Learning for Self-adaptive Information Systems

Cited by 36 publications

References 28 publications

Dynamical Orchestration and Configuration Services in Industrial IoT Systems: An Autonomic Approach

Dynamical Orchestration and Configuration Services in Industrial IoT Systems: An Autonomic Approach

Realizing self-adaptive systems via online reinforcement learning and feature-model-guided exploration

A Reference Model for Big Data Technologies

Contact Info

Product

Resources

About