Background: Due to the large variability in the prostate gland of different patient groups, manual segmentation is time-consuming and subject to inter-and intra-reader variations. Hence, we propose a U-Net model to automatically segment the prostate and its zones, including the peripheral zone (PZ), transitional zone (TZ), anterior fibromuscular stroma (AFMS), and urethra on the MRI [T2-weighted (T2W), diffusionweighted (DWI), and apparent diffusion coefficient (ADC)], and multimodality image fusion.Methods: A total of 91 eligible patients were retrospectively identified; 50 patients were considered for training process in a 10-fold cross-validation fashion and 41 ones for external test. Firstly, images were registered, and cropping was performed through a bounding box. In addition to T2W, DWI, and ADC separately, fused images were used. We considered three combinations, including T2W+DWI, T2W+ADC, and DWI+ADC, using wavelet transform. U-Net was applied to segment the prostate and its zones, AFMS, and urethra in a 10-fold cross-validation fashion. Eventually, dice score (DSC), intersection over union (IoU), precision, recall, and Hausdorff distance (HD) were used to evaluate the proposed model.Results: Using T2W images alone on the external test images, higher DSC, IoU, precision, and recall was achieved than the individual DWI and ADC images.