Learning the IPA market with individual and social rewards

Abstract

Market-based mechanisms offer a promising approach for distributed resource allocation. In this paper we consider the Iterative Price Adjustment, a pricing mechanism that can be used in commodity-market resource allocation systems. We address the scenario where agents use utility functions to describe preferences in the allocation and learn demand functions optimized for the market by Reinforcement Learning. In particular, we investigate reward functions based on the individual utilities of the agents and the Social Welfare of the market. We also evaluate the quality of demand functions obtained throughout the learning process with the aim of analyzing its influence on the behavior of the agents and exploring how much learning is enough, so the amount required can be reduced. This investigation shows that both reward functions deliver similar results when the market consists of only learning agents. We further investigate this behavior and present its theoretical-experimental explanation.