In the current era of precision oncology, it is widely acknowledged that CRC is a heterogeneous disease entity. Tumor location (right- or left-sided colon cancer or rectal cancer) is a crucial factor in determining disease progression as well as prognosis and influences disease management. In the last decade, numerous works have reported that the microbiome is an important element of CRC carcinogenesis, progression and therapy response. Owing to the heterogeneous nature of microbiomes, the findings of these studies were inconsistent. The majority of the studies combined colon cancer (CC) and rectal cancer (RC) samples as CRC for analysis. Furthermore, the small intestine, as the major site for immune surveillance in the gut, is understudied compared to the colon. Thus, the CRC heterogeneity puzzle is far from being solved, and more research is necessary for prospective trials that separately investigate CC and RC. Our prospective study aimed to map the colon cancer landscape using 16S rRNA amplicon sequencing in biopsy samples from the terminal ileum, healthy colon tissue, healthy rectal tissue and tumor tissue as well as in preoperative and postoperative stool samples of 41 patients. While fecal samples provide a good approximation of the average gut microbiome composition, mucosal biopsies allow for detecting subtle variations in local microbial communities. In particular, the small bowel microbiome has remained poorly characterized, mainly because of sampling difficulties. Our analysis revealed the following: (i) right- and left-sided colon cancers harbor distinct and diverse microbiomes, (ii) the tumor microbiome leads to a more consistent cancer-defined microbiome between locations and reveals a tumor microbiome–ileal microbiome association, (iii) the stool only partly reflects the microbiome landscape in patients with CC, and (iv) mechanical bowel preparation and perioperative antibiotics together with surgery result in major changes in the stool microbiome, characterized by a significant increase in the abundance of potentially pathogenic bacteria, such as Enterococcus. Collectively, our results provide new and valuable insights into the complex microbiome landscape in patients with colon cancer.