BackgroundHigh dose rate (HDR) brachytherapy is commonly used to treat prostate cancer. Existing HDR planning systems solve the dwell time problem for predetermined catheters and a single energy source.PurposeAdditional degrees of freedom can be obtained by relaxing the catheters' pre‐designation and introducing more source types, and may have a dosimetric benefit, particularly in improving conformality to spare the urethra. This study presents a novel analytical approach to solving the corresponding HDR planning problem.MethodsThe catheter and dual‐energy source selection problem was formulated as a constrained optimization problem with a non‐convex group sparsity regularization. The optimization problem was solved using the fast‐iterative shrinkage‐thresholding algorithm (FISTA). Two isotopes were considered. The dose rates for the HDR 4140 Ytterbium (Yb‐169) source and the Elekta Iridium (Ir‐192) HDR Flexisource were modeled according to the TG‐43U1 formalism and benchmarked accordingly. Twenty‐two retrospective HDR prostate brachytherapy patients treated with Ir‐192 were considered. An Ir‐192 only (IRO), Yb‐169 only (YBO), and dual‐source (DS) plan with optimized catheter location was created for each patient with N catheters, where N is the number of catheters used in the clinically delivered plans. The DS plans jointly optimized Yb‐169 and Ir‐192 dwell times. All plans and the clinical plans were normalized to deliver a 15 Gy prescription (Rx) dose to 95% of the clinical treatment volume (CTV) and evaluated for the CTV D90%, V150%, and V200%, urethra D0.1cc and D1cc, bladder V75%, and rectum V75%. Dose‐volume histograms (DVHs) were generated for each structure.ResultsThe DS plans ubiquitously selected Ir‐192 as the only treatment source. IRO outperformed YBO in organ at risk (OARs) OAR sparing, reducing the urethra D0.1cc and D1cc by 0.98% () and 1.09% () of the Rx dose, respectively, and reducing the bladder and rectum V75% by 0.09 () and 0.13 cubic centimeters (cc) (), respectively. The YBO plans delivered a more homogenous dose to the CTV, with a smaller V150% and V200% by 3.20 () and 1.91 cc (), respectively, and a lower CTV D90% by 0.49% () of the prescription dose. The IRO plans reduce the urethral D1cc by 2.82% () of the Rx dose compared to the clinical plans, at the cost of increased bladder and rectal V75% by 0.57 () and 0.21 cc (), respectively, and increased CTV V150% by a mean of 1.46 cc () and CTV D90% by an average of 1.40% of the Rx dose (). While these differences are statistically significant, the clinical differences between the plans are minimal.ConclusionsThe proposed analytical HDR planning algorithm integrates catheter and isotope selection with dwell time optimization for varying clinical goals, including urethra sparing. The planning method can guide HDR implants and identify promising isotopes for specific HDR clinical goals, such as target conformality or OAR sparing.