Although immune checkpoint inhibitors (ICIs) have greatly improved the prognosis of some cancer patients, the majority still fail to respond adequately, and the available biomarkers cannot reliably predict drug efficacy. The gut microbiota has received widespread attention among the various intrinsic and extrinsic factors contributing to drug resistance. As an essential regulator of physiological function, the impact of gut microbiota on host immunity and response to cancer therapy is increasingly recognized. Several studies have demonstrated significant differences in gut microbiota between responders and nonresponders. The gut microbiota associated with better clinical outcomes is called ‘favorable gut microbiota’. Significantly, interventions can alter the gut microbiota. By shifting the gut microbiota to the ‘favorable’ one through various modifications, preclinical and clinical studies have yielded more pronounced responses and better clinical outcomes when combined with ICIs treatment, providing novel approaches to improve the efficacy of cancer immunotherapy. These findings may be attributed to the effects of gut microbiota and its metabolites on the immune microenvironment and the systemic immune system, but the underlying mechanisms remain to be discovered. In this review, we summarize the clinical evidence that the gut microbiota is strongly associated with the outcomes of ICI treatment and describe the gut microbiota characteristics associated with better clinical outcomes. We then expand on the current prevalent modalities of gut microbiota regulation, provide a comprehensive overview of preclinical and clinical research advances in improving the therapeutic efficacy and prognosis of ICIs by modulating gut microbiota, and suggest fundamental questions we need to address and potential directions for future research expansion.