Perturbations of inertial confinement fusion hotspots from spherical symmetry cause an increase in the implosion velocity required for ignition, as investigated analytically by [R. Kishony and D. Shvarts, Phys. Plasmas 8, 4925 (2001)] and in numerical studies by many authors. In this paper, we analyse the mechanisms behind this effect by comparing fully 3D fluid simulations of National Ignition Facility targets to a novel analytic model of the thermal energy balance of the hotspot. The analytic model takes into account the radial variation of the state variables within the hotspot and provides an accurate relationship between the hotspot's 0D parameters (ρc, Tc, R, uR, and q) and its heating and cooling rates. The dominant effect of perturbations appears to be an increase in the inflow velocity at the hotspot's surface due to transverse flow of material between perturbation structures, causing premature thermalisation of kinetic energy before the hotspot is fully compressed. In hotspots with a broad perturbation spectrum, thermalisation of energy is inhibited by nonradial motion introduced by mode-mode interaction, reducing the yield further.