Corresponding to the atmospheric transmission windows of the electromagnetic spectrum in the low terahertz range, the mode coupling and dispersion characteristics of two helically corrugated waveguides (HCW) in the frequency ranges of 90 GHz–100 GHz and 260 GHz–265 GHz are studied. Through analytic calculations and numerical simulations, dispersion curves and structural parameters of the two frequency ranges waveguides are obtained. A novel method was proposed to obtain the dispersion of the HCW from the eigenwave solution using a periodic boundary condition. The HCW in a frequency range of 90 GHz–100 GHz was fabricated and its dispersion performance was measured. By comparing the measured results with the theoretical and the simulated results, the validity of the analytical and simulation method is verified. Limited to our machining capability, the dispersion of the 260 GHz–265 GHz HCW was only simulated and calculated and it was found that the results agree well with each other.