Acarbose (ACB) seems to be an effective drug in the management of cardiovascular risk factors. However, no previous meta-analysis of randomized controlled trials (RCTs) has been done to evaluate the effects of ACB on cardiovascular risk factors on impaired glucose tolerance (IGT), type 2 diabetes mellitus (T2D), and type 1 diabetes mellitus (T1D). We comprehensively searched electronic databases including Scopus, Web of Science, and PubMed for RCTs for related keywords up to September 2022. A random-effects model was used to estimate the weighted mean difference (WMD) and 95% confidence interval (CI). The pooled analysis demonstrated that ACB treatment had a significant effect on fasting blood glucose (FBG) (WMD = −3.55 mg/dL; 95%CI: −6.29, −0.81; p = 0.011), fasting insulin (WMD = −6.73 pmoL/L; 95%CI: −10.37, −3.10; p < 0.001), HbA1c [WMD = −0.32%; 95%CI: −0.45, −0.20; p < 0.001], body weight (WMD = −1.25 kg; 95%CI: −1.79, −0.75; p < 0.001), body mass index (BMI) (WMD = −0.64 kg/m2; 95%CI: −0.92, −0.37; p < 0.001), tumor necrosis factor-alpha (TNF-α) (WMD = −2.70 pg/mL, 95%CI: −5.25, −0.16; p = 0.037), leptin (WMD = −1.58 ng/mL; 95%CI: −2.82, −0.35; p = 0.012), alanine transaminase (ALT) (WMD = 0.71 U/L; 95%CI: −0.31, 1.85; p = 0.164), triglyceride (TG) (WMD = −13.89 mg/dL; 95%CI: −20.69, −7.09; p < 0.001), total cholesterol (TC) (WMD = −2.26 mg/dL; 95%CI: −4.18, −0.34; p = 0.021), systolic blood pressure (SBP) (WMD = −1.29 mmHg; 95%CI: −2.44, −0.15; p = 0.027), and diastolic blood pressure (DBP) (WMD = 0.02 mmHg; 95%CI: −0.41, 0.45; p = 0.925) in an intervention group, compared with a placebo group. The non-linear dose–response analysis showed that ACB reduces the TC in trial duration by >50 weeks, and 180 mg/day is more effective for the decrement of CRP. ACB can improve lipid profiles, glycemic indices, anthropometric indices, and inflammatory markers in T2D, T1D, and IGT patients.