Background: Arterial stiffness influences the prognosis of patients with end-stage kidney disease; however, the factors that promote arterial stiffness in chronic kidney disease (CKD) patients remain unknown. We aimed to explore the clinical factors associated with arterial stiffness in CKD. Methods: Between September 2017 and September 2022, all CKD patients treated at the Department of Nephrology, General Hospital of the Chinese People’s Liberation Army, excluding dialysis patients, were screened and their medical records within the last month were collected. Arterial stiffness was measured by the augmentation index (AIx). The correlative clinical factors with arterial stiffness were explored in different linear regression models. Results: 559 patients were included in the study. AIx@75 increased as the deterioration of CKDG1–CKDG5, with values of 1 (−9, 11), 5.5 (−4, 13.25), 9 (0, 16), 12 (1.5, 23.5), and 22 (13, 28), respectively (Z = 63.03, p < 0.001). Multivariate linear regression analysis showed that AIx@75 was positively associated with female sex (β = 8.926, 95% confidence interval (CI) 6.291, 11.562, p < 0.001), age (β = 0. 485, 95% CI 0.39, 0.58, p < 0.001), mean arterial pressure (MAP) (β = 0.255, 95% CI 0.159, 0.35, p < 0.001), and was negatively associated with ACEI/ARB (β = −4.466, 95% CI −6.963, −1.969, p < 0.001) and glucocorticoid (β = −3.163, 95% CI −6.143, −0.183, p = 0.038). Smoking, eGFR, hemoglobin, and cause of disease were associated with AIx@75 in multivariate linear regression models when considering factors partly. Conclusions: Female, age, smoking, MAP, eGFR, cause of disease, ACEI/ARB, and glucocorticoid were found to be associated with atherosclerosis in CKD patients.