Equilibrium in misspecified Markov decision processes

Abstract

We provide an equilibrium framework for modeling the behavior of an agent who holds a simplified view of a dynamic optimization problem. The agent faces a Markov decision process, where a transition probability function determines the evolution of a state variable as a function of the previous state and the agent's action. The agent is uncertain about the true transition function and has a prior over a set of possible transition functions; this set reflects the agent's (possibly simplified) view of her environment and may not contain the true function. We define an equilibrium concept and provide conditions under which it characterizes steady‐state behavior when the agent updates her beliefs using Bayes' rule.

Pathwise concentration bounds for Bayesian beliefs

“…Theorem 3 extends Theorem 1 to beliefs that result from observing a Markov process whose transition probabilities are unknown. This complements recent work by Molavi (2019) and Esponda and Pouzo (2021), which studied analogs of Berk (1966) and Esponda and Pouzo (2016) for Markovian environments.…”