(1) Background: The presence of adenomyosis among pregnant patients has been associated with a higher incidence of miscarriage and pregnancy complications. Although the role of adenomyosis in women undergoing in vitro fertilization (IVF) was investigated in several studies and demonstrated a potentially detrimental effect on live birth rates following IVF, most of them were small studies in which the adenomyosis diagnosis was not confirmed based on solid ultrasonographic criteria. (2) Methods: 3503 patients undergoing their first blastocyst frozen transfer through a hormonal replacement (HRT) FET cycle. Among them, 140 women had a confirmed diagnosis of adenomyosis based on the MUSA criteria. (3) Results: Adenomyosis patients were more likely to proceed with deferred FET compared with no-adenomyosis women (p = 0.002) and were significantly more likely to be treated with GnRH agonist pre-treatment (2 months) (p < 0.001). The presence of adenomyosis significantly decreased the clinical pregnancy rates (aOR 0.62, 95% CI: 0.39–0.98, p = 0.040) and live birth rates (aOR 0.46, 95% CI: 0.27–0.75, p = 0.003) and significantly increased the miscarriage rates (aOR 2.13, 95% CI: 0.98–4.37, p = 0.045). Multivariable logistic regression adjusting for age, autologous or donor oocytes, PGT-A, deferred FET, serum progesterone levels the day before FET, GnRH agonist pre-treatment, number of embryos transferred, and adenomyosis demonstrated that the use of the GnRH agonist protocol did not decrease or increase the miscarriage rate, clinical pregnancy rate, or live birth rate. (4) Conclusions: The presence of adenomyosis had a significant negative impact on the clinical outcomes of patients undergoing FET and was associated with higher miscarriage, lower clinical pregnancy, and live birth rates. GnRH agonist pre-treatment does not appear to improve clinical outcomes.