Frailty is characterized by reduced physiological reserves across multiple systems. In patients with frailty, oncological surgery has been associated with a high rate of postoperative complications and worse overall survival. Further, given that cancer and frailty can co-exist in the same patient, cancer and cancer-related symptoms can rapidly accelerate the progression of baseline frailty, which we have termed “cancer frailty”. This distinction is clinically meaningful because the prioritization of interventions and the treatment outcomes may differ based on health conditions. Specifically, in patients with cancer frailty, improvements in frailty may be achieved via surgical removal of tumors, while prehabilitation may be less effective, which may in turn result in delayed treatment and cancer progression. In this review, we focused on challenges in the surgical treatment of non-metastatic colorectal cancers in patients with frailty, including those related to decision making, prehabilitation, and surgery. Potential recommendations for treating patients with cancer frailty are also discussed.