Pollution is a critical concern of modern society for its heterogeneous effects on human health, despite a widespread lack of awareness. Environmental pollutants promote several pathologies through different molecular mechanisms. Pollutants can affect the immune system and related pathways, perturbing its regulation and triggering pro-inflammatory responses. The exposure to several pollutants also leads to alterations in gut microbiota with a decreasing abundance of beneficial microbes, such as short-chain fatty acid-producing bacteria, and an overgrowth of pro-inflammatory species. The subsequent intestinal barrier dysfunction, together with oxidative stress and increased inflammatory responses, plays a role in the pathogenesis of gastrointestinal inflammatory diseases. Moreover, pollutants encourage the inflammation-dysplasia-carcinoma sequence through various mechanisms, such as oxidative stress, dysregulation of cellular signalling pathways, cell cycle impairment and genomic instability. In this narrative review, we will describe the interplay between pollutants, gut microbiota, and the immune system, focusing on their relationship with inflammatory bowel diseases and colorectal cancer. Understanding the biological mechanisms underlying the health-to-disease transition may allow the design of public health policies aimed at reducing the burden of disease related to pollutants.