To meet the ever-increasing demand for next-generation lithography machines, the actuator plays an important role in the achievement of high acceleration of the wafer stage. However, the voice coil motor, which is widely used in high-precision positioning systems, is reaching its physical limits. To tackle this problem, a novel way to design the actuator using the magnetoresistance effect is argued due to the high force densities. However, the strong nonlinearity limits its application in the nan-positioning system. In particular, the hysteresis is coupled with eddy effects and displacement, which lead to a rate-dependent and displacement-dependent hysteresis effect in the reluctance actuator. In this paper, a Hammerstein structure is used to model the rate-dependent reluctance actuator. At the same time, the displacement-dependent of the model is regarded as the interference with the system. Additionally, a control strategy combining inverse model compensation and the disturbance observer-based discrete sliding mode control was proposed, which can effectively suppress the hysteresis effect. It is worthy pointing out that the nonlinear system is transformed into a linear system with inversion bias and disturbance by the inverse model compensation. What is more, the sliding mode controller based on the disturbance observer is designed to deal with the unmodeled dynamics, displacement disturbances, and model identification errors in linear systems. Thus, the tracking performance and robustness to external disturbances of the system are improved. The simulation results show that it is superior to the PI controller combined with an inverse compensator and even to the discrete sliding mode controller connected with inverse compensator, confirming the effectiveness of the novel control method in alleviating hysteresis.