Background: Poor glycemic control prior to cancer diagnosis for patients with pre-existing type 2 diabetes (T2DM) may predict a worse cancer diagnosis. We investigated the association between pre-diagnosis glycemic control and all-cause mortality in patients with T2DM who develop cancer. Methods: This prospective cohort study linked data from three sources covering 1989–2019: a T2DM benchmarking database, the Netherlands Cancer Registry, and the Personal Records Database. We included patients with T2DM and incident primary breast, colorectal, or prostate cancer (stage 0–III), with target glycemic control defined according to Dutch guidelines. Analysis involved estimating the association between glycemic control and all-cause mortality with Cox proportional hazard models, accounting for individual expected survival relative to the general population and relevant disease (e.g., diabetes duration and medications) and individual (e.g., age and gender) characteristics. Results: Of the 71,648 linked cases, 620 had breast cancer, 774 had colorectal cancer, and 438 had prostate cancer, with follow-up data available for 6.4 (4.2–8.4), 5.6 (2.7–7.6), and 6.3 (4.5–8.2) years, respectively. Compared to patients with pre-diagnosis glycemic control at target, the hazard ratios and 95% confidence intervals for mortality among those with pre-diagnosis glycemic control not at target were 1.40 (1.00–1.96) for breast cancer, 1.45 (1.12–1.88) for colorectal cancer, and 1.39 (0.98–1.98) for prostate cancer. Conclusions: Among patients with T2DM in Dutch primary care, poor glycemic control before diagnosis with breast and colorectal cancer can increase mortality compared to good control. Impact: Glycemic control prior to cancer diagnosis is of prognostic value.