Cultural reinforcement learning: a framework for modeling cumulative culture on a limited channel

Abstract

Humans' capacity for cumulative culture is remarkable: we can build up vast bodies of knowledge over generations. Communication, particularly via language, is a key component of this process. Previous work has described language as enabling posterior passing, where one Bayesian agent transmits a posterior distribution to the next. In practice, we cannot exactly copy our beliefs into the minds of others---we must communicate over the limited channel language provides. In this paper, we analyze cumulative culture as Bayesian reinforcement learning with communication over a rate-limited channel. We implement an agent that solves a crafting task and communicates to the next agent by approximating the optimal rate-distortion trade-off. Our model produces documented effects, such as the benefits of abstraction and selective social learning. It also suggests a new hypothesis: selective social learning can be harmful in tasks where initial exploration is required.