Recoverability of parameters from learning models

Abstract

Econometric estimation of underlying parameters of learning models has been shown to be problematic (Salmon (2001)). We delve into the causes of that phenomenon. We do so by means of two complementary investigations, using data generated in bidding problems (first price sealed bid auction and Becker-DeGroot-Marschak) by Experience Weighted Attraction (EWA) simulated agents (with true parameters thus known to the researchers). First, we set out to determine how much each of the parameters in EWA affects the agent's attraction to possible strategies, holding constant the random draws involved in the environment in which the agent operates. For the first price sealed bid auction there are large regions of the strategy space where the underlying parameters do not significantly affect the graph of attractions. Our findings regarding the recoverability of EWA parameters in Becker-DeGroot-Marschak are even less promising: large parameter changes are needed for any change in the representation of attractions to register as statistically significant. Second, a further way of examining the sources of potential difficulties in accurately estimating EWA parameters is to see whether one can reject the similarity of attractions that are all generated by the same underlying EWA parameters (but in an environment where the random draws earlier held constant are now redrawn in every trial). In Becker-DeGroot-Marschak, we find that for all but a handful of parameter combinations, one would (incorrectly) reject a null that the same parameters generated the data. A similar statement holds true for the first price sealed bid auction for parameter combinations involving δ near 1, or δ near 1, or both.