Reinforcement learning with utility-aware agents for market-based resource allocation

Gomes, Eduardo Rodrigues; Kowalczyk, Ryszard

doi:10.1145/1329125.1329247

Cited by 5 publications

(15 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This amount of learning was selected based on the visual analysis of the learnt demand functions and is the same as in [16,17]. The evaluations are made with the trends of the actual demand functions learnt by the agents.…”

Section: Market Resultsmentioning

confidence: 99%

“…In this section we compare the results of experiments for the individual and the social reward functions using the same amount of learning applied in [16,17]. We consider the case of a single IPA market with one type of resource, e.g.…”

Section: Learning the Ipa Marketmentioning

confidence: 99%

“…Our earlier works [16,17] evaluate the approach in two differ- ent cases. In one case the agents learn in the presence of agents with pre-defined static demand functions.…”

Section: Introductionmentioning

confidence: 99%

“…The next section presents the background information on the IPA Market with RL. Section 3 shows the comparison of the individual and the social reward functions using the same amount of learning we have used in our previous works [16,17]. In Section 4, demand functions obtained throughout the learning process are evaluated.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Learning the IPA market with individual and social rewards

Gomes¹,

Kowalczyk

2009

Web Intelligence and Agent Systems: An International Journal

Self Cite

View full text Add to dashboard Cite

Market-based mechanisms offer a promising approach for distributed resource allocation. In this paper we consider the Iterative Price Adjustment, a pricing mechanism that can be used in commodity-market resource allocation systems. We address the scenario where agents use utility functions to describe preferences in the allocation and learn demand functions optimized for the market by Reinforcement Learning. In particular, we investigate reward functions based on the individual utilities of the agents and the Social Welfare of the market. We also evaluate the quality of demand functions obtained throughout the learning process with the aim of analyzing its influence on the behavior of the agents and exploring how much learning is enough, so the amount required can be reduced. This investigation shows that both reward functions deliver similar results when the market consists of only learning agents. We further investigate this behavior and present its theoretical-experimental explanation.

show abstract

Section: Market Resultsmentioning

confidence: 99%

Section: Learning the Ipa Marketmentioning

confidence: 99%

“…Our earlier works [16,17] evaluate the approach in two differ- ent cases. In one case the agents learn in the presence of agents with pre-defined static demand functions.…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning the IPA market with individual and social rewards

Gomes¹,

Kowalczyk

2009

Web Intelligence and Agent Systems: An International Journal

Self Cite

View full text Add to dashboard Cite

show abstract

“…Many researchers have demonstrated experimental success for market-based planners (e.g., [18,9,19,21,10,15]). While these works experimentally validate the usefulness of their approaches and implement distributivity, only a few provide guarantees of optimality or approximate optimality (e.g., [14,21]).…”

Section: Related Workmentioning

confidence: 99%

No-Regret Learning and a Mechanism for Distributed Multiagent Planning

Calliess¹,

Gordon²

2008

View full text Add to dashboard Cite

We develop a novel mechanism for coordinated, distributed multiagent planning. We consider problems stated as a collection of single-agent planning problems coupled by common soft constraints on resource consumption. (Resources may be real or fictitious, the latter introduced as a tool for factoring the problem). A key idea is to recast the distributed planning problem as learning in a repeated game between the original agents and a newly introduced group of adversarial agents who influence prices for the resources. The adversarial agents benefit from arbitrage: that is, their incentive is to uncover violations of the resource usage constraints and, by selfishly adjusting prices, encourage the original agents to avoid plans that cause such violations. If all agents employ no-regret learning algorithms in the course of this repeated interaction, we are able to show that our mechanism can achieve design goals such as social optimality (efficiency), budget balance, and Nash-equilibrium convergence to within an error which approaches zero as the agents gain experience. In particular, the agents' average plans converge to a socially optimal solution for the original planning task. We present experiments in a simulated network routing domain demonstrating our method's ability to reliably generate sound plans.

show abstract