This study aimed to investigate the roughage tolerance of different breeds of pigs. Mashen (MS; n = 80) and Duroc × Landrace × Yorkshire (DLY; n = 80) pigs with an initial body weight of 20 ± 0.5 kg were randomly allotted to four diet treatments (n = 20 of each breed) with different fiber levels. The dietary fiber levels increased by adding 0-28% soybean hull to replace corn and soybean meal partially. According to the neutral detergent fiber (NDF) level, all treatments were MS_9N (9% NDF), MS_13.5N (13.5% NDF), MS_18N (18% NDF), MS_22.5N (22.5% NDF), DLY_9N (9% NDF), DLY_13.5N (13.5% NDF), DLY_18N (18% NDF), and DLY_22.5N (22.5% NDF). The growth performance, nutrient digestibility, intestinal morphology, and colonic short-chain fatty acids of pigs were measured. The colonic microbiota and metabolome were analyzed using 16S rDNA gene sequencing and UHPLC-MS/MS. The average daily gain and daily feed intake of MS_18N and DLY_13.5N were increased compared with MS_9N and DLY_9N, respectively (P < 0.05). The digestibility of NDF and acid detergent fiber of MS_18N were greater than that of MS_9N (P < 0.05). The villus height /crypt depth (V/C) of the duodenum, jejunum, and ileum of MS_18N and MS_22.5N increased compared with MS_9N (P < 0.05), and the V/C of duodenum and ileum of DLY_22.5N decreased compared with DLY_9N (P < 0.05). The colonic acetic acid and butyric acid concentrations of MS_18N were greater than those of MS_9N and MS_13.5N (P < 0.05). The concentrations of acetic acid and butyric acid of DLY_13.5N increased compared with DLY_9N (P < 0.05). Prevotellaceae_NK3B31_group in MS_18N and Methanobrevibacter in MS_22.5N increased compared with other groups (P < 0.05). Increasing the NDF level in diets changed the lipid and amino acid metabolism pathways. In conclusion, appropriate fiber levels can promote pigs’ growth performance and intestinal development. The optimum fiber level of the MS pig was 18% NDF, while that of the DLY pig was 13.5%. This result indicates that MS pigs had strong fiber fermentation ability due to the higher abundance of the colonic microbiota that could fully ferment fiber and provide extra energy to MS pigs.