Purpose: Growing evidence supports the efficacy and safety of high-dose-rate (HDR) brachytherapy as a boost or monotherapy in prostate cancer treatment. We initiated a new HDR prostate brachytherapy practice in April 2014. Here, we report the learning experiences, short-term safety, quality, and outcome. Methods and Materials: From April 2014 to December 2017, 164 men were treated with HDR brachytherapy with curative intent. Twenty-eight men (17.1%) underwent HDR brachytherapy as monotherapy, receiving 25 to 27 Gy in 2 fractions. Men treated with HDR brachytherapy as a boost received 19 to 21 Gy in 2 fractions. Fifty-two men (31.7%) had high-risk disease. HDR procedure times, dosimetry, and response were recorded and analyzed. Genitourinary (GU) and gastrointestinal (GI) toxicities were recorded according to the toxicity criteria of the Radiation Therapy Oncology Group. Results: Mean HDR procedure times decreased yearly from 179 minutes in 2014 to 115 minutes in 2017. Median follow-up was 18.6 months (range, 3-55 months). At last review, 79% of patients reported returning to baseline GU status, and 100% of patients noted no change in GI status from their baseline. Four patients experienced acute urinary retention. Treatment planning target volume (PTV) was defined as prostate with margins. Dosimetrically, 97.5% of all HDR implants had PTV D90 100%, 81.5% had PTV V100 95%, 73.6% had maximal urethral doses 120%, and 77.5% had rectal 1 mL dose 70% (all but one 10.8 Gy). The estimated 3-year overall survival was 98.7% (95% confidence interval, 91.4%-99.8%), and disease-free survival was 96.2% (95% confidence interval, 89.5%-98.7%). Conclusions: The low incidence of GU and GI complications in our cohort demonstrates that a HDR brachytherapy program can be successfully developed as a treatment option for patients with localized prostate cancer.