Novel treatment regimens are required for castration-resistant prostate cancers (CRPCs) that become unresponsive to standard treatments, such as docetaxel and enzalutamide. Histone deacetylase (HDAC) inhibitors showed promising results in hematological malignancies, but they failed in solid tumors such as prostate cancer, despite the overexpression of HDACs in CRPC. Four HDAC inhibitors, vorinostat, pracinostat, panobinostat and romidepsin, underwent phase II clinical trials for prostate cancers; however, phase III trials were not recommended due to a majority of patients exhibiting either toxicity or disease progression. In this review, the pharmacodynamic reasons for the failure of HDAC inhibitors were assessed and placed in the context of the advancements in the understanding of CRPCs, HDACs and resistance mechanisms. The review focuses on three themes: evolution of androgen receptor-negative prostate cancers, development of resistance mechanisms and differential effects of HDACs. In conclusion, advancements can be made in this field by characterizing HDACs in prostate tumors more extensively, as this will allow more specific drugs catering to the specific HDAC subtypes to be designed.