The occurrence and development of type 2 diabetes mellitus (T2DM) are closely related to gut microbiota. Jiaotai pill (JTP) is used to treat type 2 diabetes mellitus, with definite efficacy in clinical practice. However, it is not clear whether the therapeutic effect is produced by regulating the changes in gut microbiota and its metabolism. In this study, T2DM rat models were established by a high-fat diet and low-dose streptozotocin (STZ). Based on the pharmacodynamic evaluation, the mechanism of JTP in the treatment of type 2 diabetes mellitus was investigated by fecal metabolism and 16S rRNA gene sequencing. The results showed that JTP decreased blood glucose (FBG, HbA1c) and blood lipid (TC, TG, and LDL) levels and alleviated insulin resistance (FINS, IL-10) in T2DM rats. 16S rRNA gene sequencing results revealed that JTP increased microbiota diversity and reversed the disorder of gut microbiota in T2DM rats, and therefore achieved the therapeutic effect in T2DM. JTP regulated 13 differential flora, which were Actinobacteria, Bacteroidetes, Firmicutes, Proteobacteria, Eubacteriaceae, Prevotellaceae, Ruminococcaceae, Clostridium_IV, Clostridium_XlVa, Eubacterium, Fusicatenibacter, Romboutsia, and Roseburia. Metabolomics analysis showed that JTP interfered with 13 biomarkers to play a therapeutic role in type 2 diabetes mellitus. They were L-Valine, Choline, L-Aspartic acid, Serotonin, L-Lysine, L-Histidine, 3-Hydroxybutyric acid, Pyruvic acid, N-Acetylornithine, Arachidonic acid, L-Tryptophan, L-Alanine, and L-Methionine. KEGG metabolic pathway analysis of the above differential metabolites and gut microbiota by using the MetaboAnalyst database and Picrust software. It was found that JTP treated type 2 diabetes mellitus by affecting metabolic pathways such as amino acid metabolism, carbohydrate metabolism, and lipid metabolism. Spearman correlation analysis revealed high correlations for 7 pharmacological indicators, 12 biomarkers, and 11 gut microbiota. In this study, the therapeutic effect and potential mechanism of JTP on type 2 diabetes mellitus were preliminarily demonstrated by gut microbiota and metabolomics, which could provide a theoretical basis for the treatment of T2DM with JTP.