Immunotherapy utilizing chimeric antigen receptor T cell (CAR-T) therapy holds promise for hematologic malignancies, however, response rates and associated immune-related adverse effects widely vary among patients. Here we show, by comparing diversity and composition of the gut microbiome during different CAR-T therapeutic phases in the clinical trial ChiCTR1800017404, that the gut flora characteristically differs among patients and according to treatment stages, and might also reflect patient response to therapy in relapsed/refractory multiple myeloma (MM; n = 43), acute lympholastic leukemia (ALL; n = 23) and non-Hodgkin lymphoma (NHL; n = 12). We observe significant temporal differences in diversity and abundance of Bifidobacterium, Prevotella, Sutterella, and Collinsella between MM patients in complete remission (n = 24) and those in partial remission (n = 11). Furthermore, we find that patients with severe cytokine release syndrome present with higher abundance of Bifidobacterium, Leuconostoc, Stenotrophomonas, and Staphylococcus, which is reproducible in an independent cohort of 38 MM patients. This study has important implications for understanding the biological role of the microbiome in CAR-T treatment responsiveness of hematologic malignancy patients, and may guide therapeutic intervention to increase efficacy. The success rate of CAR-T cell therapy is high in blood cancers, yet individual patient characteristics might reduce therapeutic benefit. Here we show that therapeutic response in MM, ALL and NHL, and occurrence of severe cytokine release syndrome in multiple myeloma are associated with specific gut microbiome alterations.