To date, substantial evidence has shown a significant association between inflammatory bowel diseases (IBD) and development of colitis-associated cancer (CAC). The incidence/prevalence of IBD is higher in western countries including the US, Australia, and the UK. Although CAC development is generally characterized by stepwise accumulation of genetic as well as epigenetic changes, precise mechanisms of how chronic inflammation leads to the development of CAC are largely unknown. Preceding intestinal inflammation is one of the major influential factors for CAC tumorigenesis. Mucosal immune responses including activation of aberrant signaling pathways both in innate and adaptive immune cells play a pivotal role in CAC. Tumor progression and metastasis are shaped by a tightly controlled tumor microenvironment which is orchestrated by several immune cells and stromal cells including macrophages, neutrophils, dendritic cells, myeloid derived suppressor cells, T cells, and myofibroblasts. In this article, we will discuss the contributing factors of epithelial as well as immune cell signaling in initiation of CAC tumorigenesis and mucosal immune regulatory factors in the colonic tumor microenvironment. In depth understanding of these factors is necessary to develop novel anti-inflammatory and anti-cancer therapies for CAC in the near future.