The inductive displacement sensor is widely used in active magnetic bearing (AMB) systems to detect rotor displacement in real time, and the performance of the sensor directly affects the performance of AMB. At present, most theoretical studies on the working principle of inductive displacement sensor are based on a traditional mathematical model, ignoring the influence of the core magnetic resistance and core eddy current, which will lead to a certain error between the theoretical analysis of the sensor output characteristics and the actual situation. In this regard, based on the theory of electromagnetic field and circuit, an improved theoretical model of the inductive sensor was established in this paper by introducing the complex permeability, by which the influence of core eddy current on magnetic field can be taken into account. In order to verify the improved model, an eight-pole radial self-inductive displacement sensor with an air gap of 1 mm was designed. Then the electromagnetic field of the designed sensor was simulated by a finite element software and the GW LCR-6100 measuring instrument was used to measure the changes of the inductance and resistance of the designed sensor core coils with the rotor displacement at 20–100 kHz. The results demonstrated that there is a good linear relationship between the impedance change of the sensor coils and the rotor displacement within the measurement range of −0.4 ~ +0.4 mm. At the same time, compared with the traditional model, the sensitivity of the improved theoretical model is closer to the results from FEM and experiment, and the accuracy of the sensitivity of the improved theoretical model can be approximately doubled, despite there are certain differences with the experimental situation. Therefore, the improved theoretical model considering complex permeability is of great significance for studying the influence of core eddy current on the coil impedance of sensor.