The stability of the output speed of a hydraulic system has a great influence on the working performance of hydraulic equipment. It changes with the system working conditions. The increase in leakage caused by the wear of the hydraulic kinematic pair and the slip of the motor lead to the instability of the output speed. Although the hydraulic system can satisfy the output requirements of the active control scheme with ex ante decision making or the passive feedback control strategy with ex post compensation, it also causes an increase in system complexity and manufacturing cost. The speed stiffness as a basic characteristic of the output of the hydraulic system has not been sufficiently investigated and evaluated. In this paper, the IFMDH (inverter-fed, motor-driven hydraulic) system is taken as the object, and the coupling relationship of each link of the system is revealed by mathematical modelling. The reliability of the model is verified under a wide range of speed and load variations in combination with experiments. By redefining the speed stiffness quantification method, the effects of load conditions, motor stiffness, and speed ratio at the output end on the speed stability of the system are discussed in conjunction with the system coupling mechanism model. The conclusions show that the motor stiffness and the addition of a speed reducer have a significant effect on the system speed stiffness, where changing the output speed ratio has a significant effect on the speed stiffness. The conclusions of the study provide technical support for the rapid design, selection, and system optimisation of hydraulic systems in common scenarios.