The lubrication characteristics of a cylindrical gear transmission with a variable hyperbolic circular-arc-tooth-trace (VH-CATT) are important theoretical bases for the tribology design of the gear and its fatigue life prediction. Existing lubrication characteristics studies of it are based on simplified linear contact models, which are limited to one specific meshing position only. Therefore, the mathematical models for tooth contact analysis (TCA) and loaded tooth contact analysis (LTCA) are established to calculate most time-varying parameters in a meshing period, such as the kinematic parameters, spatial geometric parameters, and contact parameters, are calculated. Accordingly, a thermo-elastohydrodynamic lubrication (TEHL) model of an elliptical contact of this gear transmission is presented. Using all the proposed mathematical models, the TEHL results with smooth and rough tooth surfaces in a meshing period are analyzed. The obtained results reveal that these models can be used to predict the distribution of the film thickness, pressure, and temperature rise on the smooth and rough tooth surfaces of VH-CATT cylindrical gear transmission in each contact area of a meshing period. And it shows that the time-varying parameters have great influences on lubrication performance. Furthermore, the lubrication mechanism in a meshing period is investigated systematically in VH-CATT cylindrical gear transmission.