We present a combined experimental and theoretical study of the effects of pressure on Tc of the hexagonal layered superconductors nH -CaAlSi (n = 1, 5, 6), where nH labels the different stacking variants that were recently discovered. Experimentally, the pressure dependence of Tc has been investigated by measuring the magnetic susceptibility of single crystals up to 10 kbar. In contrast to previous results on polycrystalline samples, single crystals with different stacking sequences display different pressure dependences of Tc. 1H-CaAlSi shows a decrease of Tc with pressure, whereas 5H and 6H-CaAlSi exhibit an increase of Tc with pressure. Ab-initio calculations for 1H, 5H and 6H -CaAlSi reveal that an ultrasoft phonon branch associated to out-of-plane vibrations of the Al-Si layers softens with pressure, leading to a structural instability at high pressures. For 1H-CaAlSi the softening is not sufficient to cause an increase of Tc, which is consistent with the present experiments, but adverse to previous reports. For 5H and 6H the softening provides the mechanism to understand the observed increase of Tc with pressure. Calculations for hypothetical 2H and 3H stacking variants reveal qualitative and quantitative differences.