A new compound planetary gear set was produced for hybrid transmission with advantages superior to classical Ravigneaux planetary gears. However, vibration and noise problems need more attention in order to meet the market demand. Vibration characteristics of the new power-split hybrid transmission with a four-shaft compound planetary gear set are investigated. An equivalent method for stepped shaft is proposed to simplify the motor E1 shaft. Based on six degrees of freedom of helical gear pair model and shaft-bearing model, a dynamic model for the planetary mechanism is established in pure electric driving mode. The numerical analysis of the distinctive natural frequencies and corresponding vibration modes is produced. Moreover, in order to identify the noise source, noise and vibration experiments are carried out on test rig. By comparing the numerical results and test data, the noise and vibration at a frequency of 696 Hz are evidently caused by the transverse vibration of motor E1 shaft, and the rotational vibration of ring gear exacerbates the vibration and noise. Not only the impact of torques from motors E1, E2 and semi-shaft but also the influences of carrier speed and preload on differential bearings are investigated.