Background: Motherhood has been associated with reduced leisure-time physical activity (LTPA) levels. We aim to assess maternal LTPA patterns from preconception to 4 years postpartum. Methods: The study was developed with mothers from the Pelotas 2015 Birth Cohort, Southern Brazil (n = 4273). LTPA data were self-reported for preconception, antenatal, and 3, 12, 24, and 48 months postpartum. Information on LTPA referring to preconception (3 mo before pregnancy) and each gestational trimester was retrospectively collected in perinatal. The prevalence of active mothers was calculated according to recommendations (≥150 min/wk). We described changes in LTPA between follow-ups and estimated the probability of mothers being active according to previous LTPA. We used group-based trajectory analysis to identify patterns of LTPA throughout the follow-ups. Results: The prevalence of active mothers during leisure time decreased in pregnancy and postpartum compared with preconception. At 12 months postpartum, LTPA levels rose again, but at 48 months, the rates of active mothers still had not returned to preconception levels. Prior LTPA engagement was an important determinant of the maintenance of LTPA. Trajectory analysis identified 4 patterns of maternal LTPA, and ∼80% of mothers were allocated to the “always inactive” group. Higher education and income, working outside the home, multiple births, and participating in the PAMELA study intervention were associated with a higher probability of mothers not being included in this predominantly inactive group. Conclusions: Motherhood is associated with declining rates of women meeting physical activity recommendations during pregnancy and early postpartum.