This study investigated the dietary supplementation of tea residue fermented by Bacillus subtilis, Aspergillus niger, and Saccharomyces cerevisiae, to explore its effects on growth performance, digestion performance, meat quality, serum antioxidant capacity, and intestinal morphology in pigs bred for rapid growth, also known as fatteners. One hundred and ninety-two healthy “Duroc × Landrace × Yorkshire” ternary hybrid pigs (body weight 70 ± 1.0 kg) were randomly divided into four groups according to the feeding test requirements, with four replicates in each group, and 12 fatteners per replicate. The control group (CG) was fed the basal diet. Treatments 1 (T1), 2 (T2), and 3 (T3), comprising ratios of 10%, 15%, and 20% of tea residue were added to the basal diet. The test period was 60 days. The results showed that supplementation of FTR in fatteners’ diets increased final body weight (FBW), average daily gain (ADG), and feed conversion ratio (FCR) in the T1 and T2 groups (p < 0.05). Compared with the other groups, the lightness (L*) and pH were significantly affected in the T2 group (p < 0.05). Compared with the CG, dietary supplementation of FTR significantly increased the nutrient digestibility of crude protein (CP), ether extract (EE), calcium (Ca), and phosphorus (P), improved the lipase and trypsin activities, and reduced drip loss and the shear force of fatteners (p < 0.05). Glutathione peroxidase (GSH-Px) and total antioxidant capacity (T-AOC) were significantly increased in the T2 and T3 groups compared with the other groups (p < 0.05). Supplementation of FTR in the jejunum significantly increased the villi height of the T2 group and the ratio of villi height to crypt depth of the FTR groups. Compared with the other two groups, the T2 and T3 groups significantly reduced the ratio of the villous height to crypt depth in the duodenum (p < 0.05). In conclusion, the tea residue after fermentation was shown to have beneficial effects on the fattening performance, digestion performance, meat quality, serum antioxidant capacity, and intestinal morphology of fatteners.