To improve the control accuracy and robustness of variable frequency pump-controlled motor systems, a mathematical model was built to depict the system pulsation caused by the non-uniform magnetic field of a permanent magnet synchronous motor (PMSM), the harmonic current of an inverter and the structure of a plunger pump. At the same time, system pulsations were mitigated by a novel control algorithm based on sliding mode control (SMC) and harmonic suppression compensation. In addition, the performance of the system was analysed under variable load and speed. Under low-speed and heavy-load conditions, the electromagnetic torque of the PMSM and the outlet pressure of the plunger pump fluctuated greatly by traditional controlling, with maximum deviations of 11.719 N•m and 4.63 MPa, respectively. By coordinating SMC and harmonic suppression compensation, the control algorithm reduced the electromagnetic torque and the outlet pressure of the plunger pump to 45% and 40% of the traditional pump-controlled motor system driven by variable frequency PMSM values, respectively. Furthermore, the pulsation coefficient of the PMSM output speed and the hydraulic motor output speed were both controlled within 0.2%.