The third-order nonlinear susceptibility χ (3) occurs universally in materials and can provide label-free fingerprints of materials' electronic, vibrational, and structural information. One quantitative spectroscopic method to access low-energy resonances of χ (3) is the terahertz electric-field-induced second harmonic generation (TEFISH), which is particularly suitable for centrosymmetric materials without second-order processes. However, using TEFISH to measure the χ (3) spectra requires light sources with high spectral intensity between 5 and 15 THz and is more challenging precisely because of the phonon bands in many materials. Here, for the first time, we report phase-sensitive heterodyne TEFISH microscopy offering simultaneous temporal, spectral, and spatial resolution, incorporating an intense and frequency-tunable narrowband terahertz source by chirped pulse difference frequency generation in the frequency range of 4−18 THz. We demonstrated time-resolved hyperspectral TEFISH microscopy in polymer thin films (SU-8), 2D crystalline semiconductors (MoS 2 ), and subwavelength photonic resonators. By interfering with the nonlinear emission with a local oscillator field, we quantitatively retrieved the frequency, amplitude, and relative phase of the χ (3) spectra. TEFISH microscopy allows time-resolved imaging of vibrational and photonic resonances with subwavelength resolution and higher sensitivity compared to linear Fourier transform infrared spectroscopy, as well as versatility for samples in different environments by avoiding near-field probes.