Exposures to heavy metals and metalloids have been associated with decreased fecundity and fertility in couples conceiving via assisted reproduction. Heavy metals and metalloids can alter the homeostasis of critical hormones controlling sexual maturation by binding to critical hormones and receptors. This may disrupt the time course of sexual maturation directly or indirectly affecting reproductive competence in males and females. The present review aims to provide a summarized overview of associations between heavy metal exposure, reproductive concerns, and IVF outcomes. A systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) in Google Scholar, Scopus, EMBASE and PubMed databases. Initial search produced 1,351 articles from which 30 articles were eligible to be included in the systematic review. From our results, 16 articles reported associations between selected heavy metals and IVF outcomes, while 14 articles summarized the role of heavy metals in reproductive concerns. For the studies on IVF outcomes, different human samples were examined for heavy metals. Heavy metals and metalloids (Pb, Hg, Cd, Cr, Mn, As) correlated negatively with oocyte fertilization/pregnancy rates in hair, follicular fluid, serum, urine and seminal plasma samples, while Cd and Hg in whole blood samples showed no associations. For the studies on reproductive concerns, high levels of heavy metals/metalloids were implicated in the following conditions: infertility (Cd, Pb, Ba, U), spontaneous abortion/miscarriage (Pb, Cd, Sb), congenital heart disease (Al, Mg, Cd), PCOS (As, Cd, Hg, Pb), endometriosis (Pb) and uterine leiomyomata (Hg). Taken together, the results of our study suggest that the impact of heavy metals and metalloids exposure on reproductive health may contribute to the failure rates of in vitro fertilization.