A portable impedance analyzer (PIA) was developed based on a TiePie-HS3 device to provide the comparable impedance measurement accuracy of the Agilent 4294a impedance analyzer in the frequency range of 0~250 kHz. Then the PIA was applied to monitor the tensile stress-induced variation of the eddy current sensor’s impedance in a medium-carbon steel sample. A model of equivalent magnetic field induced by the elastic stress and the number of pinning sites indicated that the inductance of the eddy current loop firstly increased with the increase in the tensile stress and then decreased at the yield point of the material. The experimental results testified that the variation of impedance amplitude, the variation of phase angle, and the shift of two featured frequencies demonstrated opposite variation trends before and after the yield point, as predicated by the model. A new parameter, which combined the impedance variation information of the selected two frequencies, was found to exhibit nearly monotonous dependency on the tensile stress in elastic and plastic stages. The new parameter together with the developed portable impedance analyzer provided the solution to identify the elastic and plastic behaviors in ferromagnetic materials in practical applications with an eddy current technique.