The association between dietary diversity and frailty remains unknown in older people. We evaluated whether a limited dietary variety is associated with frailty in older adults with diabetes mellitus (DM). This cross-sectional investigation included 1357 adults (median age: 77 years, women: 61.3%). DM was determined by self-reporting, the Dietary Variety Score (DVS) was used to evaluate dietary variety, and the revised Japanese version of the Cardiovascular Health Study criteria evaluated frailty. Participants were divided into 4 groups: no DM/high DVS (non-DMHV), no DM/low DVS (non-DMLV), DM/high DVS (DMHV), and DM/low DVS (DMLV). The prevalence of frailty in each group was 3.6%, 6.7%, 6.7%, and 12.2%. After adjusting for covariates, logistic regression analysis revealed the highest odds ratio (OR) of frailty in the DMLV (non-DMLV, OR = 2.18 (95% confidence interval (CI): 1.25–3.83); DMHV, OR = 1.87 (95% CI: 0.63–5.52); DML, OR = 5.03 (95% CI: 2.05–12.35)). Another logistic regression analysis revealed that a low DVS and DM were independently associated with frailty. Both a low dietary variety and DM were independently related to frailty in older people and the combination increased the prevalence of frailty. These findings suggest that high dietary variety could be important for the prevention of frailty in people with DM.