Spatial patterns of body size are associated with environmental factors like temperature, seasonality, precipitation, resources, and altitude, among the most relevant. Latitudinal size patterns have been extensively studied. Studies on altitudinal gradients of body size, however, are relatively few. Besides, it is still unclear whether there is variation in the relationship between environmental factors and body size of species, or correlations within different altitudes. China provides a good opportunity to assess the altitudinal gradients of body size, because the entire terrain of China changes from high altitude in the west to low altitude in the east, with three natural terrain stages (low: less than 660 m.a.s.l.; medium: 660–2600 m.a.s.l.; and high: more than 2600 m.a.s.l.). Similarly, lizards are distributed across a wide altitudinal range (from −154.31 m to ~6000 m above sea level). We examined 211 lizard species in China and assessed the altitudinal gradients of Chinese lizard body sizes at assemblage and interspecific levels. Piecewise structural equation models (pSEM) were used to investigate the direct and indirect relationships between body size and environmental factors within different terrains (low, medium, and high altitudes). Overall, we found that body size in Chinese lizards decreases with increasing altitude. However, there were variations in the altitudinal gradients and drivers within the three terrain stages. Altitude has both direct (medium and low stages) and indirect (high stages) effects in shaping lizard size. We established that body size increased with increasing altitudes in the low and medium terrain stages. Seasonality had different effects on body size in medium (positively) and low (negatively) stages. However, among the three terrain stages, temperature was consistently negatively correlated with lizard size. Overall, we identified complex patterns and drivers of altitudinal gradients of body size within the three terrain stages, probably due to the different environmental conditions across altitudes.