Patient-reported outcome measures (PROMs) are advocated for the monitoring of toxicity after radiotherapy. However, studies comparing physician- and patient-reported toxicity show low concordance. In this study, we compared physician- and patient-reported toxicity in long-term prostate cancer survivors after radiotherapy, and we determined the correlation with a presumable risk factor for late toxicity: γ-H2AX foci decay ratio (FDR). Patients formerly included in a prospective study were invited to participate in this new study, comprising one questionnaire and one call with a trial physician assistant. Concordance was calculated for seven symptoms. Gamma-H2AX FDRs were determined in ex vivo irradiated lymphocytes in a previous analysis. Associations between FDR and long-term prevalence of toxicity were assessed using univariable logistic regression analyses. The 101 participants had a median follow-up period of 9 years. Outcomes were discordant in 71% of symptomatic patients; in 21%, the physician-assessed toxicity (using CTCAE) was higher, and, in 50%, the patients reported higher toxicity. We did not find a correlation between presence of toxicity at long-term follow-up and FDR. In conclusion, patients assigned greater severity to symptoms than the trial physician assistant did. Consideration of both perspectives may be warranted to provide the best care.