The objective of this study was to review publications assessing cognitive functioning in patients with prostate cancer treated with androgen deprivation therapy. We conducted a systematic review of the literature published in PubMed, Embase, Web of Science, Cochrane Library, and PsycINFO up to February 2020. A total of 31 studies were included. Half of the studies (n = 16) demonstrated that androgen deprivation therapy in patients with prostate carcinoma did not result in a negative effect on cognitive functioning, however, still a substantial proportion of the studies (n = 11) reported a negative effect on cognitive functioning. In four studies the results were inconclusive. In the three studies using additional functional magnetic resonance imaging, no significant effect on neuropsychological tests was found, but grey matter volume, brain activity, and brain connectivity were affected. Given the substantial number of studies showing a significant negative effect of androgen deprivation therapy on cognitive functioning, clinicians should be aware of this side effect. Furthermore, future research should focus on the further examination of brain characteristics using functional magnetic resonance imaging, since these techniques might be more sensitive in detecting brain abnormalities as a result of androgen deprivation therapy.