The prostate tissue structure is complex, the shape and size change is relatively large, and the surrounding anatomical structure is complex, so the task of segmenting prostate and prostate cancer is somewhat challenging. In this paper, the idea of deformable convolution is combined with the U-net algorithm widely used in medical image segmentation. By using the deformable convolution module at a specific position in the ordinary U-Net network structure, additional offsets can be added to the convolution operator and the spatial sampling position can be changed by learning the offset of the target segmentation area. The fixed receptive field of the traditional convolution operator is shifted to an adaptive receptive field that can feel the change of features, and the segmentation accuracy of the target area is improved. Experiments show that the algorithm can improve the accuracy of prostate segmentation. In this paper, the segmentation model trained with healthy prostate data is transferred to the prostate cancer data set for secondary training by simulating the way doctors read pictures. Experiments show that the segmentation effect of the lesion area is significantly improved compared with the network model trained directly with small sample prostate cancer data. The research results can provide further exploration ideas for the application of medical domain knowledge in deep learning models.