Gestational diabetes mellitus (GDM) is diagnosed by an oral glucose tolerance test (oGTT), preferably performed at 24 + 0–28 + 6 gestational weeks, and is considered a risk factor for type 2 diabetes (T2DM). In this study, we aimed to evaluate the risk of T2DM associated with abnormal oGTT performed after 28 weeks. We conducted a retrospective cohort study that included parturients with available glucose levels during pregnancy and up to 5 years of follow-up after pregnancy. Data were extracted from the computerized laboratory system of Meuhedet HMO and cross-tabulated with the Israeli National Registry of Diabetes (INRD). The women were stratified into two groups: late oGTT (performed after 28 + 6 weeks) and on-time oGTT (performed at 24 + 0–28 + 6 weeks). The incidence of T2DM was evaluated and compared using univariate analysis followed by survival analysis adjusted to confounders. Overall, 78,326 parturients entered the analysis. Of them, 6195 (7.9%) performed on-time oGTT and 5288 (6.8%) performed late oGTT. The rest—66,846 (85.3%)—had normal glucose tolerance. Women who performed late oGTT had lower rates of GDM and T2DM. However, once GDM was diagnosed, regardless of oGTT timing, the risk of T2DM was increased (2.93 (1.69–5.1) vs. 3.64 (2.44–5.44), aHR (95% CI), late vs. on-time oGTT, p < 0.001 for both). Unlike in oGTT performed on time, one single abnormal value in late oGTT was not associated with an increased risk for T2DM.