There is ongoing debate regarding the significance of complete or near-complete response after neoadjuvant chemoradiotherapy (CRT) for rectal cancer. This study assessed the prognostic value of the Dworak tumor regression grade (TRG) following neoadjuvant CRT and surgery primarily in patients with pathological stage (ypStage) II and III rectal cancer. The records of 331 patients who underwent neoadjuvant CRT followed by total mesorectal excision between 2004 and 2015 were retrospectively reviewed. Patients were categorized as having a good response (GR, TRG 3/4, n = 122) or a poor response (PR, TRG 1/2, n = 209). At a median follow-up of 65 months, five-year disease-free survival (DFS) was higher in the GR group than in the PR group (91.3% vs. 66.6%, p < 0.001). Patients with a GR and ypStage II disease had a five-year DFS that was indistinguishable from that of patients with ypStage 0–I disease (92.3% vs. 90.7%, p = 0.885). Likewise, patients with a GR and ypStage III disease had a five-year DFS similar to those with ypStage II disease (76.0% vs. 75.9%, p = 0.789). A new modified staging system that incorporates grouped TRG (GR vs. PR) was developed. The prognostic performance of this modified stage and the ypStage was compared with the Harrell C statistic. C statistic of the modified stage was higher than that of the ypStage (0.784 vs. 0.757, p = 0.012). The results remained robust after multivariate Cox regression analyses. In conclusion, a GR to neoadjuvant CRT is an independent predictor of good DFS and overall survival and further stratifies patients so as to estimate the risk of recurrence and survival among patients with ypStage II and III rectal cancer.