Background: Adjuvant chemotherapy for stage II and III rectal cancer patients is a matter of discussion. The aim of the present study was to evaluate the prognostic value of lymphovascular (LVI) and perineural (PNI) invasion in stage II rectal cancer on a national level. Materials and methods: Clinico-pathological factors associated with disease-free survival (DFS) and time to recurrence in stage II rectal cancer patients were analyzed from patient data registered in the Swedish Colorectal Cancer Registry between 2006 and 2012. Results: Of 2649 patients with TNM stage II disease, 1395 (53%) received preoperative radiotherapy and 456 (17%) preoperative chemoradiotherapy. LVI and PNI were detected in 387 (15%) and 269 (10%) patients, respectively. Adjuvant chemotherapy was planned in 14%, but more often if LVI or PNI was detected (25% and 31%, respectively, p < .001 for both). The three-year DFS and time to recurrence were 78% and 17%, respectively. Both LVI and PNI indicated worse outcome. In patients not receiving postoperative chemotherapy, the risks of recurrence after three years were 20% if LVI was seen and 22% if PNI was detected (p < .001 for both). In the absence of LVI and PNI, it was 13% and 12%, respectively. In a multivariate Cox regression analysis, patients with LVI (hazard ratio 1.44, 95% CI 1.09-1.90; p ¼ .011) and PNI (hazard ratio 1.80, 95% CI 1.34-2.43, p < .001) had significantly increased risks of recurrence. Conclusions: Stage II rectal cancer patients with LVI and PNI have an increased risk of recurrence which emphasizes the need to properly evaluate the role of adjuvant chemotherapy particularly in these subgroups.