Numerous studies have investigated the associations between serum vitamin D or testosterone and diabetes; however, inconsistencies are observed. Whether there is an interaction between vitamin D and testosterone and whether the lipid profile (total cholesterol (TC), triglyceride (TG), high-density lipoprotein cholesterol (HDL-C), and low-density lipoprotein cholesterol (LDL-C)) mediates the association between vitamin D and diabetes is unclear. To investigate the effect of vitamin D and testosterone on impaired fasting glucose (IFG) or type 2 diabetes mellitus (T2DM), 2659 participants from the Henan Rural Cohort were included in the case-control study. Generalized linear models were utilized to estimate associations of vitamin D with IFG or T2DM and interactive effects of vitamin D and testosterone on IFG or T2DM. Principal component analysis (PCA) and mediation analysis were used to estimate whether the lipid profile mediated the association of vitamin D with IFG or T2DM. Serum 25(OH)D3, 25(OH)D2, and total 25(OH)D levels were negatively correlated with IFG (odds ratios (ORs) (95% confidence intervals (CIs)): 0.99 (0.97, 1.00), 0.85 (0.82, 0.88), and 0.97 (0.96, 0.98), respectively). Similarity results for associations between serum 25(OH)D2 and total 25(OH)D with T2DM (ORs (95%CIs): 0.84 (0.81, 0.88) and 0.97 (0.96, 0.99)) were observed, whereas serum 25(OH)D3 was negatively correlated to T2DM only in the quartile 2 (Q2) and Q3 groups (both p < 0.05). The lipid profile, mainly TC and TG, partly mediated the relationship between 25(OH)D2 or total 25(OH)D and IFG or T2DM and the proportion explained was from 2.74 to 17.46%. Furthermore, interactive effects of serum 25(OH)D2, total 25(OH)D, and testosterone on T2DM were observed in females (both p for interactive <0.05), implying that the positive association between serum testosterone and T2DM was vanished when 25(OH)D2 was higher than 10.04 ng/mL or total 25(OH)D was higher than 40.04 ng/mL. Therefore, ensuring adequate vitamin D levels could reduce the prevalence of IFG and T2DM, especially in females with high levels of testosterone.