We explore the spacetime structure near non-extremal horizons in any spacetime dimension greater than two and discover a wealth of novel results: 1. Different boundary conditions are specified by a functional of the dynamical variables, describing inequivalent interactions at the horizon with a thermal bath. 2. The near horizon algebra of a set of boundary conditions, labeled by a parameter s, is given by the semi-direct sum of diffeomorphisms at the horizon with "spin-s supertranslations". For s = 1 we obtain the first explicit near horizon realization of the Bondi-Metzner-Sachs algebra. 3. For another choice, we find a non-linear extension of the Heisenberg algebra, generalizing recent results in three spacetime dimensions. This algebra allows to recover the aforementioned (linear) ones as composites. 4. These examples allow to equip not only black holes, but also cosmological horizons with soft hair. We also discuss implications of soft hair for black hole thermodynamics and entropy.