Background While neoadjuvant combined modality therapy (NA-CMT) is beneficial for most patients with locally advanced rectal cancer some patients may experience disease progression during treatment. The purpose of this study is to identify characteristics associated with progression during NA-CMT. Methods A single institution retrospective review of patients with stage II-III rectal cancer receiving NA-CMT was conducted from 2008-2019. Patients with incomplete or unknown NA-CMT treatment and those who received chemotherapy in addition to NA-CMT were excluded. Initial staging MRI was compared to post-operative pathology to determine progression. Definitions: responders (complete response or regression) and non-responders (stable disease or progression). Results 156 patients were included: 25 (16.1%) complete responders, 79 (50.6%) had evidence of regression, 34 (21.8%) were stable non-responders, and 18 (11.5%) were progressors. Those who progressed had worse overall survival. Factors associated with non-responders included black race (OR 4.5, 95% CI: 1.10-18.7) and increasing distance from the anal verge (OR 1.2, 95% CI: .2-2.9). Distance from the anal verge was determined via MRI. Recurrence was significantly more common among non-responders (15, 30.61%) when compared to responders (14, 13.46%), P = .012. Conclusion Patients who progress despite NA-CMT have overall worse survival compared to patients who do respond. While this study failed to identify modifiable or predictive risk factors for progression, the multivariate logistic regression model suggests that race and tumor biology may play a role in progression. Future studies should focus on early identification of patients who may not benefit from NA-CMT in an effort to develop alternative treatment algorithms.