We aimed to explore the causal association between type 2 diabetes (T2D) and increased arterial stiffness. We performed a Mendelian randomization (MR) analysis in 11,385 participants from a well-defined community study in Shanghai during 2011-2013. We genotyped 34 T2D-associated common variants identified in East Asians and created a genetic risk score (GRS). We assessed arterial stiffness noninvasively with the measurement of brachial-ankle pulse wave velocity (baPWV). We used the instrumental variable (IV) estimator to qualify the causal relationship between T2D and increased arterial stiffness. We found each 1-SD increase in T2D_GRS was associated with 6% higher risk in increased arterial stiffness (95% CI 1.01, 1.12), after adjustment of other metabolic confounders. Using T2D_GRS as the IV, we demonstrated a causal relationship between T2D and arterial stiffening (odds ratio 1.24, 95% CI 1.06, 1.47; P = 0.008). When categorizing the genetic loci according to their effect on insulin secretion or resistance, we found genetically determined decrease in insulin secretion was associated with increase in baPWV (b IV = 122.3 cm/s, 95% CI 41.9, 204.6; P = 0.0005). In conclusion, our results provide evidence supporting a causal association between T2D and increased arterial stiffness in a Chinese population.