We report on a real-time terahertz (THz) impulse ranging (IPR) system based on a combination of time-of-flight measurement of pulsed THz radiation and the asynchronous-optical-sampling (ASOPS) technique. The insensitivity of THz radiation to optical scattering enables the detection of various objects having optically rough surfaces. The temporal magnification capability unique to ASOPS achieves precise distance measurements of a stationary target at an accuracy of -551 μm and a resolution of 113 μm. Furthermore, ASOPS THz IPR is effectively applied to real-time distance measurements of a moving target at a scan rate of 10 Hz. Finally, we demonstrate the application of ASOPS THz IPR to a shape measurement of an optically rough surface and a thickness measurement of a paint film, showing the promise of further expanding the application scope of ASOPS THz IPR. The reported method will become a powerful tool for nondestructive inspection of large-scale structures.