The purpose of this study was to examine the independent and joint associations of acid-producing diets and depressive symptoms with physical health among breast cancer survivors. We studied a cohort of 2944 early stage breast cancer survivors who provided dietary, physical health, demographic, and lifestyle information at baseline, year 1, and year 4. We assessed the intakes of acid-producing diets via two commonly used dietary acid load scores: potential renal acid load (PRAL) and net endogenous acid production (NEAP). Physical health was measured using the Rand 36-Item Short Form Health Survey (SF-36), consisting of physical functioning, role limitation due to physical function, bodily pain, general health, and overall physical health subscales. Increased dietary acid load and depression were each independently and significantly associated with reduced physical health subscales and overall physical health. Further, dietary acid load and depression were jointly associated with worse physical health. For instance, depressed women with dietary acid load higher than median reported 2.75 times the risk (odds ratio = 2.75; 95% confidence interval: 2.18–3.47) of reduced physical function and 3.10 times the risk of poor physical health (odds ratio = 3.10; 95% confidence interval: 2.53–3.80) compared to non-depressed women with dietary acid load lower than median. Our results highlight the need of controlling acid-producing diets and the access of mental care for breast cancer survivors.