We derive analytic bounds on the shape of the primordial power spectrum in the context of single-field inflation. In particular, the steepest possible growth has a spectral index of n s − 1 = 4 once transients have died down. Its primary implication is that any constraint on the power spectrum at a particular scale can be extrapolated to an upper bound over an extended range of scales. This is important for models which generate relics due to an enhanced amplitude of the primordial scalar perturbations, such as primordial black holes. In order to generate them, the power spectrum needs to grow many orders of magnitude larger than its observed value on CMB scales -typically achieved through a phase of ultra slow-roll inflation -and is thus subject to additional constraints at small scales. We plot all relevant constraints including CMB spectral distortions and gravitational waves sourced by scalar perturbations at second order. We show how this limits the allowed mass of PBHs, especially for the large masses of interest following recent detections by LIGO and prospects for constraining them further with future observations. We show that any transition from approximately constant slow-roll inflation to a phase where the power spectrum rapidly rises necessarily implies an intervening dip in power. We also show how to reconstruct a potential that can reproduce an arbitrary time-varying , offering a complementary perspective on how ultra slow-roll can be achieved. * C.Byrnes@sussex.ac.uk