The incidence rate of prostate cancer is higher in male cancers. With a hidden initiation of disease and long duration, prostate cancer seriously affects men’s physical and mental health. Prostate cancer is initially androgen-dependent, and endocrine therapy can achieve good results. However, after 18–24 months of endocrine therapy, most patients eventually develop castration-resistant prostate cancer (CRPC), which becomes metastatic castration resistant prostate cancer (mCRPC) that is difficult to treat. Chimeric Antigen Receptor T cell (CAR-T) therapy is an emerging immune cell therapy that brings hope to cancer patients. CAR-T has shown considerable advantages in the treatment of hematologic tumors. However, there are still obstacles to CAR-T treatment of solid tumors because the physical barrier and the tumor microenvironment inhibit the function of CAR-T cells. In this article, we review the progress of CAR-T therapy in the treatment of prostate cancer and discuss the prospects and challenges of armed CAR-T and combined treatment strategies. At present, there are still many obstacles in the treatment of prostate cancer with CAR-T, but when these obstacles are solved, CAR-T cells can become a favorable weapon for the treatment of prostate cancer.