Advantage based value iteration for Markov decision processes with unknown rewards

Alizadeh, Pegah; Chevaleyre, Yann; Lévy, François

doi:10.1109/ijcnn.2016.7727695

Cited by 4 publications

(4 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are looking for an optimal QoS-selection strategy satisfying the user's requirements in terms of QoS attributes. Since the user's preferences with respect to QoS attributes are unknown, we use a partially known MDP model, and more particularly, a vector-valued MDP (VMDP) model [2].…”

Section: Services Composition As An Mdpmentioning

confidence: 99%

“…2) MDPs can be easily integrated into a dynamic environment with uncertainty. In this article, we use part of the results presented in [2] and [41] on how to efficiently learn the users' preferences. These results are based on the combined use of MDP and RL techniques.…”

Section: B Services Composition As a Discrete-time Vmdpmentioning

confidence: 99%

“…In this section, we introduce an algorithm with this aim, namely, interactive value iteration for services composition (IVI-SC). It is possible to use interactive value iteration methods for finding the optimal workflow by learning user's preferences weights dynamically [2], [41].…”

Section: Interactive Rl Algorithms For the Services Composition Prmentioning

confidence: 99%

“…We propose a reinforcement learning (RL)-based approach to dynamically obtain the optimal service composition, according to the user's expectations on the QoSs. Markov Decision Processes (MDP) with unknown rewards enable us to tackle this challenge by optimizing rewards according to the answered queries for any user [1], [2]. Based on our approach and according to the given theoretical results, our algorithm minimizes the number of queries needed to select the best service composition.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Reinforcement Learning for Interactive QoS-Aware Services Composition

Alizadeh

Osmani

Khanouche

et al. 2021

IEEE Systems Journal

Self Cite

View full text Add to dashboard Cite

An important and challenging research problem in web of things is how to select an appropriate composition of concrete services in a dynamic and unpredictable environment. The main goal of this article is to select from all possible compositions the optimal one without knowing a priori the users' quality of service (QoS) preferences. From a theoretical point of view, we give bounds on the problem search space. As the QoS user's preferences are unknown, we propose a vector-valued MDP approach for finding the optimal QoS-aware services composition. The algorithm alternatively solves MDP with dynamic programming and learns the preferences via direct queries to the user. An important feature of the proposed algorithm is that it is able to get the optimal composition and, at the same time, limits the number of interactions with the user. Experiments on a real-world large size dataset with more than 3500 web services show that our algorithm finds the optimal composite services with around 50 interactions with the user. Index Terms-Quality of services (QoS), reinforcement learning (RL), services composition, web of things (WoT). I. INTRODUCTION T HE web of things (WoT) opens the way for the development of new intelligent applications that can be used to provide innovative and valuable services in several domains, such as smart cities, smart homes, ambient assisted living, and connected cars [11], [35]. One of the WoT challenges is to deal with services composition to ensure suitable services flexibility and customization. This is crucial to meet the evolving needs of users, which can be expressed through several parameters of quality of service (QoS) (such as response time, throughput, availability, price, popularity, etc.). Following the web service paradigm [4] services, composition problems are specified as a workflow involving abstract and concrete services.

show abstract

Section: Services Composition As An Mdpmentioning

confidence: 99%

Section: B Services Composition As a Discrete-time Vmdpmentioning

confidence: 99%

Section: Interactive Rl Algorithms For the Services Composition Prmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Reinforcement Learning for Interactive QoS-Aware Services Composition

Alizadeh

Osmani

Khanouche

et al. 2021

IEEE Systems Journal

Self Cite

View full text Add to dashboard Cite

show abstract

SAIL: Simulation-Informed Active In-the-Wild Learning

Short

Allevato

Thomaz

2019

2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI)

View full text Add to dashboard Cite

Tacit knowledge and a multi-method approach in Asset Management

Holanda¹,

Souza²,

Adorni³

et al. 2022

logeion

View full text Add to dashboard Cite

This paper has two main objectives. The first one is to reflect on the validity of data in analysis and projections that underpin the engineering asset management of organizations, considering, on the one hand, a certain resistance or even inadequate use of data and information of a subjective nature and; on the other hand, a consolidated reliance on quantitative approaches and decisions based on data series. The second objective is to contextualize the applicability of combining qualitative data based on experts’ experience and on the tacit knowledge built in organizations, with approaches based essentially on quantitative data, according to the data availability and the decision scenario over the asset life cycle. We present two application of an approach combining quali and quanti data. One example interrelates Statistics with Psychology and the other combines elicitated data about asset indicators with parameters deterministically calculated. Both applications aiming at providing more realistic indicators and supporting the Asset Management process to make more assertive decisions.

show abstract

Advantage based value iteration for Markov decision processes with unknown rewards

Cited by 4 publications

References 8 publications

Reinforcement Learning for Interactive QoS-Aware Services Composition

Reinforcement Learning for Interactive QoS-Aware Services Composition

SAIL: Simulation-Informed Active In-the-Wild Learning

Tacit knowledge and a multi-method approach in Asset Management

Contact Info

Product

Resources

About