The aim of the study was to compare the side effects of high-dose-rate brachytherapy (HDRBT) and low-dose-rate brachytherapy (LDRBT), with a particular focus on the effects on sexual functions and sexual well-being (PROMOBRA study, NCT02258087). Localized low-risk and low-intermediate-risk prostate cancer patients were treated with mono LDR (N = 123, 145 Gy dose) or mono HDR brachytherapy (N = 117, 19/21 Gy). Prior to the treatment and during follow-up (at 3, 6, 9, 12, 18, and 24 months after treatment, and then annually after two years), patients completed patient-reported outcome measurement (PROM) questionnaires EORTC QLQ-PR-25, International Index of Erectile Function (IIEF), and IIEF-5 (SHIM). We compared the patients in different group breakdowns (HDR vs. LDR, hormone naïve and hormone-receiving HDR vs. LDR, hormone naïve and hormone-receiving patients in general, and 19 Gy HDR vs. 21 Gy HDR). In the hormone-naive LDR group, erectile function, orgasm function, sexual desire, satisfaction with intercourse, and overall satisfaction functions significantly decreased compared to baseline throughout the whole follow-up period. However, there were significant decreases in function at a maximum of three time points after HDR therapy without hormone therapy. In hormone-receiving patients, the orgasm function was significantly better in the HDR group at multiple time points compared to the baseline, and sexual desire improved at four time points. According to our results, both LDRBT and HDRBT can be safely administered to patients with localized prostate cancer. In hormone-naive patients, the HDR group showed only recovering decreases in sexual functions, while the LDR group showed a lasting decline in multiple areas. Thus, HDR appears to be more advantageous to hormone-naive patients.