Clinical research results of vitamin D supplementation in the improvement of prediabetes remain controversial. Accordingly, a literature search was conducted of PubMed, Embase (Ovid), and Web of Science prior to 9 November 2021. Randomized controlled studies reported that the following indicators were included: body mass index (BMI), fasting blood glucose (FBG), 2 h oral glucose tolerance test plasma glucose (2h-PG), hemoglobin A1c (HbA1c), insulin resistance by homeostasis model assessment (HOMA-IR), homeostasis model assessment of β-cell function (HOMA-B), and fasting insulin (FINS). Twenty-nine articles (N = 3792) were included in the present meta-analysis. Intriguingly, vitamin D supplementation resulted in a vast improvement in FBG (standardized mean difference (SMD) = −0.38; 95%CI: −0.59, −0.16), HbA1c (SMD = −0.14; 95%CI: −0.22, −0.06) and FINS (SMD = 0.18; 95%CI: −0.26, −0.09), but not in other outcomes. However, preferred changes were observed in subgroups, as follows: Asia (SMD2h-PG = −0.25, 95%CI: −0.45, −0.04), study duration ≥1 year (SMDHOMA-IR = −0.44, 95%CI: −0.81, −0.06) (SMDHOMA-B = 0.34, 95%CI: 0.01, 0.66), baseline 25(OH)D < 50 nmol/L (SMD2h-PG = −0.23, 95%CI: −0.39, −0.06), and baseline 25(OH)D ≥ 50 nmol/L (SMDHOMA-IR = −0.50, 95%CI: −0.96, −0.03). In conclusion, oral supplementation of vitamin D has shown better effects in improving FBG, HbA1c, and FINS compared with controls among prediabetics; long-term vitamin D supplementation could have additional effects in participants with vitamin D deficiency for 2h-PG, HOMA-IR, and HOMA-B.