Exposure to xenoestrogens is a probable cause of male infertility in humans. Consumption of high-fat diets and exposure to bisphenol A (BPA) is pervasive in America. Here, we test the hypothesis that gestational exposure to high dietary fats and/or BPA disrupt spermatogenesis in adulthood. Sprague-Dawley rats were fed diets containing 10 kcal% butter fat (AIN), 39 kcal% butter fat (HFB), or 39 kcal% olive oil (HFO), with or without BPA (25 μg/kg body weight/day) during pregnancy. One group of male offspring received testosterone (T)- and estradiol-17β (E2)-filled implants or sham-implants from postnatal day (PND)70-210. Another group was naturally aged to 18 months. We found that adult males with gestational exposure to BPA, HFB, or HFB + BPA, in both the aged group and the T + E2-implanted group, exhibited impairment of spermatogenesis. In contrast, gestational exposure to HFO or HFO + BPA did not affect spermatogenesis. Sham-implanted, gestational exposed groups also had normal spermatogenesis. Loss of ERα expression in round spermatids and premature expression of protamine-1 in diplotene spermatocytes were features associated with impaired spermatogenesis. Compared with the single-treatment groups, the HFB + BPA group experienced more severe effects, including atrophy.