Transition-metal interfaces and multilayers are a promising class of systems to realize nanometer-sized, stable magnetic skyrmions for future spintronic devices. For room temperature applications, it is crucial to understand the interactions which control the stability of isolated skyrmions. Typically, skyrmion properties are explained by the interplay of pair-wise exchange interactions, the Dzyaloshinskii-Moriya interaction and the magnetocrystalline anisotropy energy. Here, we demonstrate that higher-order exchange interactions – which have so far been neglected – can play a key role for the stability of skyrmions. We use an atomistic spin model parametrized from first-principles and compare three different ultrathin film systems. We consider all fourth-order exchange interactions and show that, in particular, the four-site four spin interaction has a large effect on the energy barrier preventing skyrmion and antiskyrmion collapse into the ferromagnetic state. Our work opens perspectives to stabilize topological spin structures even in the absence of Dzyaloshinskii-Moriya interaction.