Although more studies are demonstrating that a father’s environment can influence child health and disease, the molecular mechanisms underlying non-genetic inheritance remain unclear. It was previously thought that sperm exclusively contributed its genome to the egg. More recently, association studies have shown that various environmental exposures including poor diet, toxicants, and stress, perturbed epigenetic marks in sperm at important reproductive and developmental loci that were associated with offspring phenotypes. The molecular and cellular routes that underlie how epigenetic marks are transmitted at fertilization, to resist epigenetic reprogramming in the embryo, and drive phenotypic changes are only now beginning to be unraveled. Here, we provide an overview of the state of the field of intergenerational paternal epigenetic inheritance in mammals and present new insights into the relationship between embryo development and the three pillars of epigenetic inheritance: chromatin, DNA methylation, and non-coding RNAs. We evaluate compelling evidence of sperm-mediated transmission and retention of paternal epigenetic marks in the embryo. Using landmark examples, we discuss how sperm-inherited regions may escape reprogramming to impact development via mechanisms that implicate transcription factors, chromatin organization, and transposable elements. Finally, we link paternally transmitted epigenetic marks to functional changes in the pre- and post-implantation embryo. Understanding how sperm-inherited epigenetic factors influence embryo development will permit a greater understanding related to the developmental origins of health and disease.