Generally, during the accelerating process of construction machinery large inertia hydraulic swing system, a lot of kinetic energy is accumulated, and it will be eventually dissipated through the relief valve orifice during the decelerating process, which results in serious energy waste and large overflow loss. In order to solve this problem, a new dual source hydraulic motor with non-uniform flow distribution is proposed, which can drive high-frequency swing mechanism efficiently. The new dual source hydraulic motor has two groups of oil inlet and outlet, corresponding to two groups of different displacement. The two groups of inlet and outlet are respectively connected with the energy recovery unit and the main driving unit. During the starting process, the energy recovery unit assists the main driving unit to drive the swing mechanism together, so that the main drive unit output torque and overflow loss are reduced; during the braking process, the energy recovery unit recovers the braking energy. Finally, the integrated control of braking energy direct recovery and utilization is realized. In the research, the 38 t hydraulic excavator is taken as the research object. Firstly, the multi-body dynamics electro-hydraulic simulation model of the dual source integrated driving hydraulic excavator swing system is established, and the accuracy of the model is verified by experiments. Then the effects of the accumulator working pressure and the dual source hydraulic motor displacement ratio on the working performance of the system were simulated. The simulation results show that with the displacement ratio increasing, the accelerating time, the maximum angular velocity and decelerating time increase. Compared with the original system, when the displacement ratio is 0.6, the comprehensive performance of the system is the best, and the hydraulic pump output energy is reduced by 48.6% and 46.6% respectively in the process of full load and no-load operation.