This research presents a novel methodology for measuring the complex permittivity of a material under test (MUT) in a millimeter-wave (mmWave) band by using two rectangular waveguide adapters. Contrary to the conventional Nicolson-Ross-Weir (NRW) method, the proposed complex permittivity measurement method does not require a material fabrication process for exact MUT insertion into a waveguide. In our complex permittivity measurement, simple commercial waveguide adapters are employed instead of large flange structures. The proposed complex permittivity measurement of a non-destructive MUT is achieved by combining the NRW method, the Gaussian weighting moving average filtering technique, a full-wave electromagnetic analysis, and an optimization technique. Furthermore, the proposed methodology is validated by fabricating a Teflon-based MUT and by measuring the complex permittivity of the MUT in the Ka band (26.5–40 GHz). The results indicate that the proposed methodology exhibits good agreement with the data sheet.