Immunotherapy has improved patient survival in many types of cancer, but for prostate cancer, initial results with immunotherapy have been disappointing. Prostate cancer is considered an immunologically excluded or cold tumor, unable to generate an effective T-cell response against cancer cells. However, a small but significant percentage of patients do respond to immunotherapy, suggesting that some specific molecular subtypes of this tumor may have a better response to checkpoint inhibitors. Recent findings suggest that, in addition to their function as cancer genes, somatic mutations of PTEN, TP53, RB1, CDK12, and DNA repair, or specific activation of regulatory pathways, such as ETS or MYC, may also facilitate immune evasion of the host response against cancer. This review presents an update of recent discoveries about the role that the common somatic mutations can play in changing the tumor microenvironment and immune response against prostate cancer. We describe how detailed molecular genetic analyses of the tumor microenvironment of prostate cancer using mouse models and human tumors are providing new insights into the cell types and pathways mediating immune responses. These analyses are helping researchers to design drug combinations that are more likely to target the molecular and immunological pathways that underlie treatment failure.