Diabetes mellitus among older men has been associated with increased bone mineral density but paradoxically increased fracture risk. Given the interactions among medication treatment, glycemic control, and diabetes‐associated comorbidities, the relative effects of each factor remains unclear. This retrospective study includes 652,901 male veterans aged ≥65 years with diabetes and baseline hemoglobin A1c (HbA1c) value. All subjects received primary care in the Veterans Health Administration (VHA) from 2000 to 2010. Administrative data included ICD9 diagnoses and pharmacy records and was linked to Medicare fee‐for‐service data. Hazard ratios (HR) for any clinical fracture and hip fracture were calculated using competing risk hazards models, adjusted for fracture risk factors including age, race/ethnicity, body mass index (BMI), alcohol and tobacco use, rheumatoid arthritis, corticosteroid use, as well as diabetes‐related comorbidities including cardiovascular disease, chronic kidney disease, and peripheral neuropathy. HbA1c <6.5% was associated with a higher risk of any clinical fracture (HR = 1.08, 95% confidence interval [CI] 1.06–1.11) compared with the reference HbA1c of 7.5% to 8.5%. Fracture risk was not increased among those with A1c ≥8.5%, nor among those with A1c 6.5% to 7.5%. Use of insulin was independently associated with greater risk of fracture (HR = 1.10, 95% CI 1.07–1.12). There was a significant interaction between insulin use and HbA1c level, (p < 0.001), such that those using insulin with HbA1c <6.5% had HR = 1.23 and those with HbA1c 6.5% to 7.5% had HR = 1.15. Metformin use was associated with decreased fracture risk (HR = 0.88, 95% CI 0.87–0.90). We conclude that among older men with diabetes, those with HbA1c lower than 6.5% are at increased risk for any clinical and hip fracture. Insulin use is associated with higher fracture risk, especially among those with tight glycemic control. Our findings demonstrate the importance of the treatment regimen and avoiding hypoglycemia for fracture prevention in older men with diabetes. © 2019 American Society for Bone and Mineral Research.