A retrospective data analysis suggested that the levels of boar taint compounds depend on the polyunsaturated fatty acid (PUFA) level of the adipose tissue (AT) being significantly greater in the unsaturated AT. In addition, we recently reported that hydrolysable tannins (HTs) offered to entire males (EMs) reduce skatole and, to a greater extent, indole levels in the AT. Thus, the objective of the study was to determine the impact of HTs and a high dietary level of PUFA on growth performance and board taint compounds in EMs. In addition, the interaction between PUFA and HTs on gut microbiota and its link to intestinal skatole and indole production was investigated. At 25 kg BW, 44 EM originating from 11 litters were randomly assigned within litter to four dietary treatments. Two basal grower (25–60 kg BW) and finisher (60–105 kg BW) diets containing either 2% soy oil (H = high PUFA level) or 2% tallow (L = low PUFA level) were formulated. The H and L diets were either supplemented (H+/L+) or not (H−/L−) with 3% chestnut extract containing 50% HTs. The pigs had ad libitum access to the diets and were slaughtered at 170 days of age. The microbiota composition was investigated through the 16S rRNA gene sequences obtained by next-generation sequencing (Illumia MiSeq platform, San Diego, CA, USA) and analyzed with a specific packages in R, version 3.5.0. Regardless of the PUFA content, the EMs fed the H+ diets were 2% (p < 0.01) less feed efficient overall. This was due to the slower (p = 0.01) growth in the finisher period despite similar feed intake. Carcass characteristics were not affected by the diets. Regardless of HT feeding, the PUFA level in the AT of the H pigs was 10% greater (p = 0.05) than in the L pigs. The indole level tended (p = 0.08) to be 50% lower in the H+ group. Surprisingly, the pigs that were fed diet H− had greater skatole levels than those fed diet L−, with intermediate skatole levels in the H+ and L+. Independent of the PUFA level, the HTs decreased bacteria abundance and qualitatively affected the microbiota composition. In conclusion, these data do not confirm that boar taint compound levels were related to PUFA levels in the AT. However, HTs can be considered to be a promising alternative to conventional antibacterial additives, with no detrimental effects on pig gut health and with appealing properties for reducing the synthesis of the main components of boar taint.