Baihu Rensheng decoction (BHRS) can effectively improve insulin resistance (IR) and decrease blood glucose in diabetic patients. However, its specific mechanism of action remains unclear. In this study, a type 2 diabetes mellitus (T2DM) rat model was established using a high-fat diet combined with streptozotocin (STZ) injection and treated with BHRS. Firstly, the therapeutic and anti-inflammatory effects of BHRS on T2DM were evaluated. Secondly, the effects of BHRS on gut permeability were evaluated and western blot was used to detect the changes of TLR4/NF-κB pathway-related protein expressions in liver. Finally, 16S rRNA sequencing was used to detect alteration of gut microbiota diversity and abundance in rats after BHRS treatment. Our results showed that BHRS could alleviate the hyperglycemia, hyperlipidemia, IR, and pathological changes of liver, pancreas, and kidney in T2DM rats. BHRS could also decrease the levels of pro-inflammatory cytokines and inhibit the oxidative stress. Immunohistochemistry showed BHRS could increase the expression tight junction-related proteins (ZO-1 and occludin) in colon. Besides, the level of LPS in serum was decreased after BHRS treatment. Western blot results showed that the protein expression of TLR4, MyD88 and the phosphorylation IκB, and NF-κBp65 were lowered after BHRS treatment. 16S rRNA sequencing showed that BHRS treatment altered the diversity of gut microbiotra and decreases the Firmicutes/Bacteroidetes (F to B) ratio at the phylum level. At the genus level, BHRS could increase the relative abundances of Lactobacillus, Blautia, and Anaerostipes and decrease the relative abundances of Allobaculum, Candidatus Saccharimonas, and Ruminococcus. In conclusion, our study revealed the various ameliorative effects of BHRS on T2DM, including improving the liver and kidney functions and alleviating the hyperglycemia, hyperlipidemia, pathological changes, oxidative stress and inflammatory response. The mechanisms of BHRS on T2DM are likely linked to the repair of gut barrier and the inhibition of TLR4/NF-κB-mediated inflammatory response and the improvement in the dysbiosis of gut microbiota.