Neoadjuvant concurrent chemoradiotherapy (CCRT), followed by radical proctectomy, is the standard treatment for locally advanced rectal cancer. However, a poor response and therapeutic resistance continue to occur despite this treatment. In this study, we analyzed the microarray datasets (GSE68204) of rectal cancer from the Gene Expression Omnibus database, and identified CHD4 as one of the most significantly up-regulated genes among all subunits of the nucleosome remodeling and histone deacetylation (NuRD) complex, in non-responders to CCRT, among locally advanced rectal cancer (LARC) patients. We confirmed the predictive and prognostic significance of CHD4 expression in CCRT treatment, and its correlation with other clinicopathological features, such as tumor regression grade (TRG), therapeutic response, and patient survival. This was carried out by immunohistochemical studies on endoscopic biopsy tissues from 172 rectal cancer patients, receiving neoadjuvant concurrent chemoradiotherapy (CCRT). A high expression of CHD4 was significantly associated with pre-treatment tumor status (p < 0.001) and lymph node metastasis (p < 0.001), post-treatment tumor status (p < 0.001), and lymph node metastasis (p < 0.001), vascular invasion (p = 0.042), and tumor regression grade (p = 0.001). A high expression of CHD4 could also predict poor disease-specific survival and metastasis-free survival (log-rank test, p = 0.0373 and p < 0.0001, respectively). In multivariate Cox proportional-hazards regression analysis, CHD4 overexpression was an independent factor of poor prognosis for metastasis-free survival (HR, 4.575; 95% CI, 1.717–12.192; p = 0.002). By in vitro studies, based on cell line models, we also demonstrated that, the overexpression of CHD4 induced radio-resistance in microsatellite instability-high (MSI-H) colorectal cells (CRCs). On the contrary, the knockdown of CHD4 enhanced radiosensitivity in microsatellite stable (MSS) CRCs. Altogether, we have identified CHD4 as an important regulator of radio-resistance in both MSI-H and MSS CRC cell lines.