The great majority of breast and prostate tumors are hormone-dependent cancers; hence, estrogens and androgens can, respectively, drive their developments, making it possible to use pharmacological therapies in their hormone-dependent phases by targeting the levels of steroid or modulating their physiological activity through their respective nuclear receptors when the tumors relapse. Unfortunately, at some stage, both breast and prostate cancers become resistant to pharmacological treatments that aim to block their receptors, estrogen (ER) or androgen (AR) receptors, respectively. So far, antiestrogens and antiandrogens used in clinics have been designed based on their structural analogies with natural hormones, 17-β estradiol and dihydrotestosterone. Plants are a potential source of drug discovery and the development of new pharmacological compounds. The aim of this review article is to highlight the recent advances in the pharmacological modulation of androgen or estrogen levels, and their activity through their cognate nuclear receptors in prostate or breast cancer and the effects of some plants extracts.