In this paper, we demonstrate the capabilities of the terahertz pulse time-domain holography in visualisation, simulation, and assessment of broadband THz vortex beam formation dynamics upon its shaping by elements of beam converter, and further propagation and manipulation. By adding Jones matrix formalism to describe broadband optical elements, we highlight the differences in the spatial-spectral and spatio-temporal structure of the formed vortex and vector beams dependence on the modulator used and visualise their modal features. The influence of diffraction field structure from each element in the broadband vortex modulator is revealed in numerical simulation and the formed beams are analysed against the simplified Laguerre-Gaussian beam model.