Diabetes-induced cognitive impairment (DCI) presents a major public health risk among the aging population. Previous clinical attempts on known therapeutic targets for DCI, such as depleted insulin secretion, insulin resistance, and hyperglycaemia have delivered poor patient outcomes. However, recent evidence has demonstrated that the gut microbiome plays an important role in DCI by modulating cognitive function through the gut–brain crosstalk. The bioactive compound tanshinone IIA (TAN) has shown to improve cognitive and memory function in diabetes mellitus models, though the pharmacological actions are not fully understood. This study aims to investigate the effect and underlying mechanism of TAN in attenuating DCI in relation to regulating the gut microbiome. Metagenomic sequencing analyses were performed on a group of control rats, rats with diabetes induced by a high-fat/high-glucose diet (HFD) and streptozotocin (STZ) (model group) and TAN-treated diabetic rats (TAN group). Cognitive and memory function were assessed by the Morris water maze test, histopathological assessment of brain tissues, and immunoblotting of neurological biomarkers. The fasting blood glucose (FBG) level was monitored throughout the experiments. The levels of serum lipopolysaccharide (LPS) and tumor necrosis factor-α (TNF-α) were measured by enzyme-linked immunoassays to reflect the circulatory inflammation level. The morphology of the colon barrier was observed by histopathological staining. Our study confirmed that TAN reduced the FBG level and improved the cognitive and memory function against HFD- and STZ-induced diabetes. TAN protected the endothelial tight junction in the hippocampus and colon, regulated neuronal biomarkers, and lowered the serum levels of LPS and TNF-α. TAN corrected the reduced abundance of Bacteroidetes in diabetic rats. At the species level, TAN regulated the abundance of B. dorei, Lachnoclostridium sp. YL32 and Clostridiodes difficile. TAN modulated the lipid metabolism and biosynthesis of fatty acids in related pathways as the main functional components. TAN significantly restored the reduced levels of isobutyric acid and butyric acid. Our results supported the use of TAN as a promising therapeutic agent for DCI, in which the underlying mechanism may be associated with gut microbiome regulation.