Pancreatic cancer is one of the most lethal cancers, due to its uniquely aggressive behavior and resistance to therapy. The tumor microenvironment of pancreatic cancer is immunosuppressive, and attempts at utilizing immunotherapies have been unsuccessful. Radiation therapy (RT) results in immune activation and antigen presentation in other cancers, but in pancreatic cancer has had limited success in stimulating immune responses. RT activates common pathways of fibrosis and chronic inflammation seen in pancreatic cancer, resulting in immune suppression. Here we describe the pancreatic tumor microenvironment with regard to fibrosis, myeloid and lymphoid cells, and the impact of RT. We also describe strategies of targeting these pathways that have promise to improve outcomes by harnessing the cytotoxic and immune-activating aspects of RT.