Using a reinforcement-learning algorithm, we model an agent-based simulation of a public goods game with endogenous punishment institutions. We propose an outcome-based model of social preferences that determines the agent’s utility, contribution, and voting behavior during the learning procedure. Comparing our simulation to experimental evidence, we find that the model can replicate human behavior and we can explain the underlying motives of this behavior. We argue that our approach can be generalized to more complex simulations of human behavior.