Background: Little is known about how changes in a constellation of lifestyle factors affect health-related quality of life (HRQoL) in colorectal cancer (CRC) survivors. Our study aimed to investigate the association between changes in healthy lifestyle and HRQoL over time in survivors of stage I–IV CRC. Methods: We included 2,283 long-term (≥5 years postdiagnosis) survivors. A healthy lifestyle score (HLS) comprising smoking, alcohol consumption, diet, physical activity, and body fatness was derived at diagnosis and 5-year follow-up (5YFU) and categorized as low, moderate, or high. We assessed HRQoL with the EORTC Quality of Life Questionnaire-Core 30 at 5YFU and 10-year follow-up. We used multivariable linear regression and linear mixed models to explore associations between changes in HLS and HRQoL over follow-up. Results: A low baseline HLS was associated with poorer functioning and global health/QoL and a higher symptom burden at 5YFU compared with a high baseline HLS. An improved HLS from baseline to 5YFU was associated with better functioning, higher global health/QoL, and fewer symptoms at 5YFU than a maintained-high HLS. In longitudinal analyses, improved HLS was associated with better functioning at follow-up. Survivors with a maintained-high or an improved HLS reported generally less fatigue, pain, and dyspnea at follow-ups compared with survivors with a maintained-low or decreased HLS. Conclusions: Change toward a healthier lifestyle since diagnosis was associated with better HRQoL in long-term CRC survivors. Our results support the importance of maintaining and/or promoting a healthier lifestyle among CRC survivors postdiagnosis.