Predictors for postoperative recovery after colorectal cancer surgery are usually investigated in relation to length of stay (LoS), readmission, or 30-day morbidity. This study describes patient characteristics and surgery-related factors associated with patient-reported recovery 1 and 6 months after surgery. In total, 153 consecutively included patients who were recovering from colorectal cancer surgery reported their level of recovery using the Postoperative Recovery Profile. Multiple logistic regression analysis was used to calculate associations with recovery, defined as good or poor, divided into five recovery dimensions: physical symptoms, physical functions, psychological, social and activity. Better preoperative health predicted good recovery regarding three dimensions 1 month after surgery. Regarding all dimensions 1 month after surgery, poor recovery was predicted by a poor recovery on the day of discharge within corresponding dimensions. Higher age was associated with good recovery 6 months after surgery, while chemotherapy showed negative associations. Overall, a majority of factors had a negative impact on recovery, but without any obvious relation to one specific dimension or point in time. Those factors were: high Body Mass Index, comorbidity, abdominoperineal resection, loop ileostomy, colostomy and LoS. This study illustrates the complexity of postoperative recovery and a need for individualised follow-up strategies.