Objective of the present study was to evaluate the effect of vitamin D supplementation on glycose homeostasis, islet function, and diabetes progress. Literatures were searched via electronic databases, websites, and previous reviews from the earliest available time to the end of May 2020. Randomized controlled trials initially designed for diabetes and prediabetes with 25dihydroxyvitamin D [25(OH)D]<30 ng/ml were included. All data were analyzed and presented based on the Cochrane guidelines and PRISMA guidelines. In total, 27 articles (n = 1,932) were enrolled in this study. Vitamin D supplementation significantly improved fasting blood glucose, postprandial blood glucose, and quantitative insulin sensitivity check index in diabetes and prediabetes with baseline 25(OH)D<30 ng/ml. Higher percentages regressing from prediabetes to normal glucose status [1.60 (1.19, 2.17), p = 0.002, n = 564] and lower percentage progressing from prediabetes to diabetes [0.68 (0.36, 1.27), p = 0.23, n = 569] were found in the supplementation group. The positive effects of vitamin D supplementation on body mass index, waist, HDL-C, LDL-C, and CRP were also demonstrated. In conclusion, modest improvements in vitamin D supplementation on short-term glycose homeostasis, insulin sensitivity, and disease development in diabetes and prediabetes with 25(OH)D<30 ng/ml were demonstrated, but more research needs to be conducted in the future to support the clinical application. (Register ID: CRD42020186004)