Cancer affects millions of individuals worldwide. Thus, there is an increased need for the development of novel effective therapeutic approaches. Tumorigenesis is often coupled with immunosuppression which defeats the anticancer immune defense mechanisms activated by the host. Novel anticancer therapies based on the use of immune checkpoint inhibitors (ICIs) are very promising against both solid and hematological tumors, although still exhibiting heterogeneous efficacy, as well as tolerability. Such a differential response seems to derive from individual diversity, including the gut microbiota (GM) composition of specific patients. Experimental evidence supports the key role played by the GM in the activation of the immune system response against malignancies. This observation suggests to aim for patient-tailored complementary therapies able to modulate the GM, enabling the selective enrichment in microbial species, which can improve the positive outcome of ICI-based immunotherapy. Moreover, the research of GM-derived predictive biomarkers may help to identify the selected cancer population, which can benefit from ICI-based therapy, without the occurrence of adverse reactions and/or cancer relapse. The present review summarizes the landmark studies published to date, which have contributed to uncovering the tight link existing between GM composition, cancer development and the host immune system. Bridging this triangle of interactions may ultimately guide towards the identification of novel biomarkers, as well as integrated and patient-tailored anticancer approaches with greater efficacy.