The aim of this prospective cohort study was to examine the relationships between the intakes of various vitamins and the loss of muscle mass in older people with type 2 diabetes (T2DM). The change in skeletal muscle mass index (SMI, kg/m2) (kg/m2/year) was defined as follows: (SMI at baseline (kg/m2) − SMI at follow-up (kg/m2))/follow-up period (year). The rate of SMI reduction (%) was calculated as follows (the change in SMI (kg/m2/year)/SMI at baseline (kg/m2)) × 100. The rate of SMI reduction ≥ 1.2% was considered as the loss of muscle mass. Among 197 people with T2DM, 47.2% of them experienced the loss of muscle mass at the 13.7 ± 5.2 month follow-up. Vitamin B1 (0.8 ± 0.3 vs. 0.8 ± 0.3 mg/day, p = 0.031), vitamin B12 (11.2 ± 8.3 vs. 13.4 ± 7.5 μg/day, p = 0.049), and vitamin D (16.5 ± 12.2 vs. 21.6 ± 13.0 μg/day, p = 0.004) intakes in people with the loss of muscle mass were significantly lower than those without. Vitamin D intake was related to the loss of muscle mass after adjusting for sex, age, exercise, alcohol, smoking, body mass index, SMI, glucagon-like peptide-1 agonist, sodium glucose cotransporter-2 inhibitor, insulin, HbA1c, creatinine, energy intake, and protein intake (adjusted odds ratio 0.93, 95% confidence interval: 0.88–0.97, p = 0.003). This study showed that vitamin D intake was related to the loss of muscle mass in older people with T2DM. Vitamin B12 intake tended to be related to the loss of muscle mass, although vitamin A, vitamin B2, vitamin B6, vitamin C, and vitamin E intake were not related.