Prostate cancer and treatment with androgen deprivation therapy (ADT) affect significant numbers of the male population. Endocrine effects of ADT are a critical consideration in balancing the benefits and risks of treatment on long-term survival and quality of life. This review highlights the latest advances in androgen manipulation in prostate cancer with an emphasis on the effects of ADT on muscle and bone, which universally affects the health and well-being of men undergoing ADT for prostate cancer. Muscle mass declines with ADT; however, the evidence that this correlates with a decrease in muscle strength or a decrease in physical performance is discordant. Cortical bone decay also occurs in association with an increase in fracture risk, hence optimization of musculoskeletal health in men undergoing ADT is crucial. The role of exercise, and current and emerging anabolic therapies for muscle as well as various new strategies to prevent loss of bone mass in men undergoing ADT are discussed. Future well-designed, prospective, controlled studies are required to elucidate the effects of ADT on physical performance, which are currently lacking, and larger randomized controlled trials are required to test the efficacy of medical therapies and exercise interventions to target proven deficits and to ensure safety in men with prostate cancer.