We conduct a theoretical study of the propagation of few-cycle, ultrafast vortices (UFVs) carrying orbital angular momentum (OAM) in free space. Our analysis reveals much more complex temporal dynamics than that of few-cycle fundamental Gaussian-like beams, particularly when approaching the single-cycle regime and the magnitude of the topological charge l is high. The recently described lower bound |l| to the number of oscillations of UFVs with propagation-invariant temporal shape (isodiffracting UFVs) is found to hold on average also for UFVs of general type, with variations along the propagation direction above and below that bound, even vanishing locally. These variations are determined by the so-called Porras factor or g0-factor characterizing the dependence of the Rayleigh distance of the spectral constituents with frequency. With a given available bandwidth, UFVs must widen temporally with increasing magnitude of the topological charge, and must widen or may shrink temporally during propagation as a result of the axially varying, g0-dependent lower bound. Under very restrictive conditions in their generation, an UFV can be shrunk below the lower bound |l| at a focus into a kind of locally compressed state of OAM, but it broadens well-above |l| and distorts in a tiny fraction of the depth of focus because of the dispersions introduced by Gouy's phase and wave front mismatch. These propagation phenomena have implications and should be taken into account in experiments and applications of UFVs, such as the generation of high-harmonics and attosecond pulses with high OAM, or in OAM-based ultrafast communications systems, as well as in other areas of physics such as acoustics or electron waves.