The working performance and service life of the two-speed transmission system directly affects the performance and service life of helicopters and other equipment. One of the main tasks of the two-speed transmission system research is to improve its dynamic characteristics. For the two-speed transmission system in high-speed gear, a purely torsional nonlinear dynamic differential equation set considering the number of planetary gears, backlash, and clutch dynamic load is established by using the lumped parameter method, and the equations are dimensionless. Then the dimensionless differential equation set is solved by using the variable step-size fourth-order Runge–Kutta method, and the phase diagram and Poincare diagram of high-speed gear are obtained. By changing the dynamic friction coefficient of the friction clutch and the backlash of the gear pair, the influence of parameter change on the nonlinear dynamic characteristics of the system is analyzed. The results show that, with the increase of excitation frequency, the system has experienced single cycle, quasi-cycle, chaos, and double cycle, then changed from double cycle to chaotic motion, and then changed from chaotic motion to double cycle and single cycle motion in turn, and found the path to chaos. In the low-frequency band, reducing the friction coefficient of the friction clutch can reduce vibration amplitude; In the middle-frequency band, reducing the friction coefficient will make the system tend to unstable vibration. In the high-frequency band, it is a single-cycle movement, which is not affected by friction coefficient.