Terahertz (THz) spectroscopy systems are widely employed to retrieve the chemical and material composition of a sample. This is single-handedly the most important driving motivation in the field and has largely contributed to shaping THz science as an independent subject. The limited availability of high-resolution imaging devices, however, still represents a major technological challenge in this promising field of research. In this theoretical work, we tackle this challenge by developing a novel nonlinear ghost imaging approach that conceptually outperforms established single-pixel imaging protocols at inaccessible wavelengths. Our methodology combines nonlinear THz generation with time-resolved field measurements, as enabled by state-of-theart time domain spectroscopy techniques. As an ideal application target, we consider hyperspectral THz imaging of semitransparent samples with nonnegligible delay contribution, and we demonstrate how time-resolved, full-wave acquisition enables accurate spatiotemporal reconstruction of complex inhomogeneous samples.