We investigate the problem of N identical bosons that are coupled to an impurity particle with infinite mass. For non-interacting bosons, we show that a dynamical impurity-boson interaction, mediated by a closed-channel dimer, can induce an effective boson-boson repulsion which strongly modifies the bound states consisting of the impurity and N bosons. In particular, we demonstrate the existence of two universal "multi-body" resonances, where all multi-body bound states involving any N emerge and disappear. The first multi-body resonance corresponds to infinite impurity-boson scattering length, a → +∞, while the second corresponds to the critical scattering length a * > 0 beyond which the trimer (N = 2 bound state) ceases to exist. Crucially, we show that the existence of a * ensures that the ground-state energy in the multi-body bound-state region, ∞ > a > a * , is bounded from below, with a bound that is independent of N . Thus, even though the impurity can support multi-body bound states, they become increasingly fragile beyond the dimer state. This has implications for the nature of the Bose polaron currently being studied in cold-atom experiments. arXiv:1807.09948v1 [cond-mat.quant-gas]