This review article argues for a better acknowledgement by penal philosophers of the diversity of subjects, agents, and practices of punishment. Much current penal philosophy has an unhelpful hyper-focus on the criminal punishment of culpable adults, by states, often through imprisonment. This paradigmatic case is important, but other subjects, agents, and practices of punishment are not statistically insignificant side-issues, and a comprehensive account of punishment should address them. Our understanding of punishment as a whole can be enhanced by considering non-paradigmatic punishment, with implications for whether and when punishment is justified, how we should understand appropriate authority, and how we should understand and engage with abolitionist arguments. We explore non-paradigmatic penal practices (community punishments, suspended prison sentence, restorative justice, and alternative jurisprudence), non-paradigmatic punishing agents (International judicial bodies, schools, and religious communities; with practices such as boycotts, shaming and shunning) and non-paradigmatic subjects of punishment (collective agents, corporations and children).