Reinforcement learning in auction experiments

Abstract

In this paper, we study the learning behavior possibly emerging in six series of prediction market experiments. We first find, from the experimental outcomes, that there is a general positive correlation between subjects' earning performance and their reliance on using limit order to trade. Given this connection, we, therefore, focus on subjects' learning behavior in terms of their use of limit order or market order. A 3-parameter Roth-Erev reinforcement learning (RL) model is empirically constructed for each subject. A numerical algorithm known as differential evolution is applied to estimate the maximum likelihood function derived from the RL model. The results of the estimated parameters show not just their great heterogeneity, but also the sharp contrasts among subjects. This great heterogeneity is, however, not caused by the experimental designs, such as the information distribution among agents. The statistical analysis shows that the heterogeneity comes from the subject themselves. In other words, they may be considered as personality traits of subjects. We then go further to test whether these personality traits have impact on subjects' earning performance, and some initial evidences suggest the significance of both attention and determination parameters. Hence, to be winners, not only does subjects need to learn, but also they need to learn with right personality traits (parameters).