Extensive research has revealed that the score derived from the Gleason grading system plays a pivotal role in predicting prostate cancer (PCa) progression. However, the underlying involvement of Gleason-related genes in PCa requires further investigation. This study aimed to identify Gleason-related genes with the potential to guide PCa therapy and future research. Differentially expressed genes (DEGs) were identified by comparing PCa tissues with high or low Gleason scores using the Gene Expression Omnibus (GEO) and the Cancer Genome Atlas (TCGA) databases. R v3.6.1, SPSS v23, and ImageJ software were used for all analyses. An effective recurrence-free survival (RFS) predictive model based on seven Gleason-related genes was established and validated (TCGA, AUC = 0.803; five years, AUC = 0.740; three years, AUC = 0.722; one year, AUC = 0.711; GSE46602, AUC = 0.766; five years, AUC = 0.808; three years, AUC = 0.723; one year, AUC = 0.656; GSE116918, AUC = 0.788; five years, AUC = 0.704; three years, AUC = 0.693; one year, AUC = 0.996). Calibration and nomogram plots were conducted. Weighted correlation network analysis (WGCNA) was used, and COL5A2 was selected for further analysis. The results from in vitro experiments demonstrated that COL5A2 was upregulated in PCa with high Gleason scores. The knockdown of COL5A2 inhibited cell proliferation and invasion in PC-3 and LNCaP cell lines. Meanwhile, COL5A2 displayed a strong association with immune infiltration, which might be an underlying immunotherapy target for PCa. We successfully established a robust RFS predictive model. The findings from this study indicated that COL5A2 could promote cell proliferation and invasion in PCa.