Prostate cancer (PCa) is one of the principal causes of cancer-related death worldwide. The roles and mechanisms of long non-coding RNA (lncRNA) involved in the development of PCa remain incompletely understood. The present study aimed to investigate the role and mechanism of lncRNA in PCa tumorigenesis. In the present study, lncRNA cancer susceptibility candidate 11 (CASC11) was revealed to be a crucial regulator of PCa progression. The expression profiles of CASC11 in PCa were identified through analysis of The Cancer Genome Atlas and Gene Expression Omnibus datasets, and validated in human PCa specimens and cell lines. Gain-and loss-of-function assays were utilized to explore the biological role of CASC11 in PCa initiation and progression. RNA-sequencing, RNA pull-down and RNA immunoprecipitation analyses were used to explore potential mechanisms with which CASC11 may be associated. Rescue experiments were further conducted to confirm this association. The present results revealed that CASC11 was dominantly distributed in the nuclei of PCa cells, and was highly expressed in PCa tissues and cells. Overexpression of CASC11 was markedly associated with increased tumor proliferation and migratory ability. Functionally, decreased proliferation and migration, as well as inhibited xenograft tumor growth, were observed in CASC11-silenced PCa cells, whereas the opposite effects were detected in CASC11-overexpressing cells. Mechanistically, CASC11 promoted progression of the cell cycle and competitively interacted with Y-box binding protein 1 (YBX1) to block the p53 pathway. Given this, poly (β-amino ester) (PBAE)/small interfering RNA-CASC11 (si-CASC11) nanoparticles were applied to inhibit CASC11 expression and enhance the antitumor effect in vivo. The results revealed that PBAE/si-CASC11 nanoparticles augmented the antitumor efficacy of CASC11 knockdown in vivo. In conclusion, the present study suggested that CASC11 may regulate PCa progression and elucidated a novel CASC11/YBX1/p53 signaling axis, providing a potential lncRNA-directed therapeutic strategy particularly for the treatment of patients with PCa.