Pre-therapy dosimetry prediction is a prerequisite for treatment planning and personalized optimization of the emerging radiopharmaceutical therapy (RPT). Physiologically based pharmacokinetic (PBPK) model, describing the intrinsic pharmacokinetics of radiopharmaceuticals, have been proposed for pre-therapy prediction of dosimetry. However, it is restricted with organ-wise prediction and the customization based on pretherapy measurements is still challenging. On the other side, artificial intelligence (AI) has demonstrated the potential in pretherapy dosimetry prediction. Nevertheless, it is still challenging for pure data-driven model to achieve voxel-wise prediction due to huge gap between the pre-therapy imaging and post-therapy dosimetry. This study aims to integrate the PBPK model into deep learning for voxel-wise pre-therapy dosimetry prediction. A conditional generative adversarial network (cGAN) integrated with the PBPK model as regularization was developed. For proof of concept, 120 virtual patients with 68 Ga-PSMA-11 PET imaging and 177 Lu-PSMA-I&T dosimetry were generated using realistic in silico simulations. In kidneys, spleen, liver and salivary glands, the proposed method achieved better accuracy and dose volume histogram than pure deep learning. The preliminary results confirmed that the proposed PBPK-adapted deep learning can improve the pre-therapy voxel-wise dosimetry prediction and may provide a practical solution to support treatment planning of heterogeneous dose distribution for personalized RPT.