To make the gear better adapt to the high temperature environment, the effects of tooth surface friction and thermal deformation on the dynamic characteristics of the gear were studied. First, based on the basic theory of gear nonlinear dynamics, the backlash model is established considering the effects of thermal deformation. Then, an improved multi-degree-of-freedom nonlinear dynamic model of gears is established taking into account the effects of thermal deformation and friction. The effect of tooth surface friction and thermal deformation on the system motion state is analyzed. Finally, to better analyze the relationship between impact vibration and transmission efficiency, the percussion vibration index is introduced. Compared with the existing research, a gear multi-degree-of-freedom dynamic model is established by considering the influence of tooth surface friction and thermal deformation. The percussion vibration index is introduced as an indicator of gear running health. The research result can provide reference for the optimization and cooling of the gear transmission system.