Some adults with type 2 diabetes mellitus (T2DM) believe that chromium-containing supplements will help control their disease, but the evidence is mixed. This narrative review examines the efficacy of chromium supplements for improving glycemic control as measured by decreases in fasting plasma glucose (FPG) or hemoglobin A1c (HbA1c). Using systematic search criteria, 20 randomized controlled trials of chromium supplementation in T2DM patients were identified. Clinically meaningful treatment goals were defined as an FPG of 7.2 mmol/dL, a decline in HbA1c to 7%, or a decrease of 0.5% in HbA1c. In only a few randomized controlled trials did FPG (5 of 20), HbA1c (3 of 14), or both (1 of 14) reach the treatment goals with chromium supplementation. HbA1c declined by 0.5% in 5 of 14 studies. On the basis of the low strength of existing evidence, chromium supplements have limited effectiveness, and there is little rationale to recommend their use for glycemic control in patients with existing T2DM. Future meta-analyses should include only high-quality studies with similar forms of chromium and comparable inclusion/exclusion criteria to provide scientifically sound recommendations for clinicians.