The traditional loader drive system is based on the hydraulic torque converter as the key component, and its gear is shifted through the coordination of the clutch and gearbox, which greatly increases power loss and operator fatigue. To address the above problem, a variable-structure hydrostatic drive system is proposed. This system adopts a closed-loop design and adjusts the displacement of the pump and dual motors to follow the throttle opening of the vehicle, achieving automatic gear shifting and smooth speed regulation. It can also automatically change the system structure according to the vehicle’s speed, meeting the vehicle’s demand for rapid switching output of high torque and high speed. At the same time, in the displacement matching control process, an adaptive sliding mode control scheme based on radial basis function neural network compensation is proposed. This scheme designs an adaptive hyperparameter update strategy according to the characteristics of the system to effectively compensate for changes in uncertain factors. Experimental results show that, compared to traditional drive systems, this system has the characteristics of simple operation, smooth speed regulation, and high fuel economy.