Purpose We aimed to determine whether physical activity (PA) may mitigate side effects of adjuvant endocrine therapy (AET), or whether prolonged sitting may exacerbate them. AET often causes debilitating endocrine symptoms that compromise quality of life (QOL) in women diagnosed with hormone receptor positive breast cancer (BC). Methods We examined longitudinal patterns among PA and sitting with endocrine symptoms and QOL over 3 years in 554 female BC survivors undergoing AET using parallel process latent growth curve models. Results At baseline, women were a mean age of 59 years, mostly white (72%), with overweight/obesity (67%), and approximately 50% being within 1 year of diagnosis. Unconditional models showed a significant increase in PA (P<.01) over time but no change in sitting. Endocrine symptoms, general and BC-specific QOL all got significantly worse over time (P<.01). Parallel process models showed no cross-sectional or longitudinal associations between PA and endocrine symptoms, but higher PA was associated with higher QOL (P=.01). Increases in PA were not associated with changes in QOL. Conversely, higher baseline sitting time was associated with worse endocrine symptoms, general and BC specific QOL (Ps<.01), yet worse baseline endocrine symptoms were associated with a slower rate of increase in sitting (P<.01). Better baseline QOL was also associated with an increase in sitting (Ps<.01) while an increase in sitting was associated with reductions in symptoms (P=.017). Model fit statistics (x2, CFI, TLI, SRMR) were acceptable. Conclusion Both PA and sitting are important for managing symptoms and maintaining QOL in BC survivors.