Across the world, citizens sidestep the state to punish offenses on their own. Such vigilantism can help communities provide order, yet it raises concerns about public accountability and the rights of the accused. While prior research has identified the structural correlates of vigilantism, an open question is in which cases citizens prefer vigilantism over conventional policing. To make sense of these preferences, we draw on two logics of punishment: state substitution and retribution. Using survey data from a conjoint experiment presented to over 9000 households across Guatemala, we find that preferences for vigilantism depend on how transgressive the crime is as well as how unlikely it is to be prosecuted by the state. Victim and perpetrator gender, as well as crime severity and profession of the perpetrator, affect whether people endorse vigilante punishment. These results ultimately raise concerns about the viability of "informal" forms of policing.