Pathological studies on the different histological effects between neoadjuvant chemotherapy (NAC) and preoperative chemoradiation therapy (preoperative CRT) have not been performed. The purpose of this study is to elucidate the histological differences in tissue received from NAC and preoperative CRT for rectal cancer to evaluate whether a pathological assessment method used after CRT can be applied for NAC. One hundred and thirty-eight patients were enrolled in this study; 88 patients underwent their operations after preoperative CRT or NAC, and 50 patients underwent surgery only. Residual tumor area was measured using morphometry software and we compared the stromal component of myofibroblasts, immune cells, and vasculature to elucidate the difference of therapeutic effect between them. The grade of reduction after preoperative CRT was more prominent than that seen in NAC. Also, ypT downstaging was more prominent in preoperative CRT than in NAC, and ypN downstaging was more frequent in NAC than in preoperative CRT. Preoperative CRT showed more marked myofibroblasts and fewer immune cells than did NAC, which indicates different effects on the cancer microenvironment. Our histological results suggest different effects between NAC and preoperative CRT on tumor tissue. The best assessment method available for a variable therapeutic protocol should be further investigated.