Transmission efficiency is a key characteristic of Hydro-mechanical Continuously Variable Transmission (HMCVT), which is related to the performance of heavy-duty tractors. Predicting the HMCVT transmission efficiency is beneficial for the real-time adjustment of transmission ratio during heavy-duty tractor operations, so as to obtain better performance. Aiming at the problems of accurate method, low accuracy, and high noise in the prediction of HMCVT transmission efficiency, this paper proposes a method based on Variational Mode Decomposition (VMD), Particle Swarm Optimization (PSO), and Back Propagation (BP) neural networks to improve the quality of transmission efficiency prediction. Firstly, a simple theoretical model was established to obtain the influencing factors of transmission efficiency. Then, based on these factors, the transmission efficiency was tested on the bench under multiple conditions and the influence degree of each factor on transmission efficiency was divided using Partial Least Squares (PLS) method. Finally, the VMD method was used to denoise the test data, and a BP model, which was improved using the PSO method, was established to predict the processed data. The results showed that transmission efficiency of HMCVT is most affected by output speed, followed by power, and least by input speed. The VMD method can accurately extract effective signals and noise signals from the original data, and reconstruct signals, reducing the noise proportion. Using three conditions, the prediction regression accuracy of the PSO–BP model is 7.02%, 7.88%, and 9.26% higher than that of the BP model, respectively. In the three prediction experiments, the maximum differences in the MAE, the MAPE, and the RMSE of the PSO–BP model are 0.002, 0.463%, and 0.004, respectively, which are 0.006, 0.796%, and 0.003 lower than those of the BP model.