Establishing a precise system level model for transmission system is quite necessary and feasible for today's computer technology. Taking into account of the exact parameters and actual working condition, an accurate model for a new deep hybrid transmission with compound planetary gear set is constructed, which in consideration of the gravity and centrifugal effect. The system level model contains the gears, shafts, bearings, clutches, electric motors and the housings. Few studies are found to investigate structure influence in system level for noise, vibration and harshness performance. Validation for the simulation results with test in three typical working conditions is carried out, which shows a quite well agreement. Furthermore, some parameters for this compound planetary gear set are investigated. The results show that for radial clearance between the ratchet and ring gear the smaller the better, while the axial clearance between the ratchet and ring gear requires cautious consideration to realize a better vibration performance for different planetary gear set. In the case of carrier, the radius of the connection pawl and the thickness of the carrier input shaft have remarkable influence on vibration characteristics. The vibration amplitude is the lowest when these two values equaling to 55 and 7 mm, respectively. Other than that, it is interesting to notice that the carrier position has a significant influence on the system vibration in pure electric drive mode. It is shown that if the planet carrier is fixed and sun gear rotates as input, the position of carrier should keep planet gears at high gravity center and symmetrical distribution on both sides of carrier centerline in the gravitational direction. The vibrational magnitude is the smallest as the carrier rotating 50 along z axis for this compound planetary gear set.