The aim of this research was to reveal the characteristics of gut microbiome altered by acarbose intervention in Japanese patients with type 2 diabetes (T2D) and its possible association with habitual dietary intake. Eighteen patients with T2D were administered acarbose for four weeks. The abundances of two major phyla, namely Actinobacteria and Bacteroidetes, were reciprocally changed accompanied by the acarbose intervention. There were also significant changes in the abundances of ten genera, including the greater abundance of Bifidobacterium, Eubacterium, and Lactobacillus and the lower abundance of Bacteroides in the group after the intervention than that before the intervention. Hierarchical clustering of habitual dietary intake was performed based on the pattern of changes in the gut microbiota and were classified into distinct three clusters. Cluster I consisted of sucrose, cluster II mainly included fat intake, and cluster III mainly included carbohydrate intake. Moreover, the amount of change in Faecalibacterium was positively correlated with the intake of rice, but negatively correlated with the intake of bread. The intake of potato was negatively correlated with the amount of change in Akkermansia and Subdoligranulum. Acarbose altered the composition of gut microbiome in Japanese patients with T2D, which might be linked to the habitual dietary intake.