Factors affecting the blood-testis barrier function may be involved in testicular damage and male infertility. Two cytokines play an important role in the barrier regulation, namely transforming growth factor beta 3 (TGF-β3) and tumor necrosis factor (TNF-α). The aim of this study was to investigate the potential association between TGF-β3 (TGFB3) and TNF-α (TNF) gene polymorphisms and male infertility. A total of 846 subjects, 423 diagnosed with male infertility and 423 fertile men were enrolled. TGFB3 (rs2268626:T > C, rs3917158:C > T, rs2284792:A > G, rs2268625:T > C, rs3917187:C > T) and TNF (rs1800629:-308G > A) gene polymorphisms were genotyped. No association between TNF genotype and infertility was observed. As for TGFB3, the genotypes distribution was similar in infertile and fertile men. However, rs2284792 minor allele frequency was significantly higher among infertile subjects. Heterozygous rs2284792 AG genotype was associated with increased odds for infertility [OR = 1.40 (95% CI 1.05–1.86), p = 0.021] and similar results were observed for G allele carrier status [OR = 1.40 (95% CI 1.06–1.84), p = 0.017]. Heterozygosity in TGFB3 rs3917158 was also associated with the infertility [OR = 1.37 (95% CI 1.01–1.87), p = 0.041]. The TGFB3 variant genotypes were associated with lower spermatozoa motility parameters in fertile men. The results indicate that variants in TGFB3 gene may be associated with male infertility. However, the findings require further replication and validation.