(1) Background: The purpose of this article was to investigate the association between self-reported physical activity (PA) and all-cause mortality in ambulatory patients with chronic kidney disease (CKD), stage 4–5 including maintenance dialysis. (2) Methods: Ambulatory patients with CKD (eGFR < 30 mL/min/1.73 m2) with conservative treatment or chronic dialysis were included. PA was assessed using the Saltin–Grimby Physical Activity Level Scale. A Cox proportional hazards regression model––adjusted for age, sex, plasma–albumin, body mass index, socioeconomic status, and treatment––was applied. (3) Results: Participants (n = 374) were followed 39 ± 15 months from entry to death or censoring. Throughout the study period of 39 months, 156 deaths (42%) were registered. Regarding physical activity, 128 (34%) of the participants were inactive, 212 (57%) were moderately active, and 34 (9%) were highly or vigorously active. Moderate PA was associated with a decreased mortality risk in women (n = 150) compared to inactivity (HR 0.27 (0.15; 0.51), p < 0.001), whereas a high/vigorous level of PA was not significantly associated with mortality risk compared to inactivity. In men (n = 224), the associations between PA levels and mortality risk were not significant. (4) Conclusions: Moderate PA was associated with reduced all-cause mortality in ambulatory women with stage 4–5 CKD with or without maintenance dialysis treatment. Physical activity was not significantly associated with mortality in men.