In recent years, an apparent decline in human sperm quality has been observed worldwide. One in every 5.5 couples suffers from infertility, with male reproductive problems contributing to nearly 40% of all infertility cases. Although the reasons for the increasing number of infertility cases are largely unknown, both genetic and environmental factors can be contributing factors. In particular, exposure to chemical substances during mammalian male germ cell development has been linked to an increased risk of infertility in later life owing to defective sperm production, reproductive tract obstruction, inflammation, and sexual disorders. Prenatal exposure to nanomaterials (NMs) is no exception. In animal experiments, maternal exposure to NMs has been reported to affect the reproductive health of male offspring. Male germ cells require multiple epigenetic reprogramming events during their lifespan to acquire reproductive capacity. Given that spermatozoa deliver the paternal genome to oocytes upon fertilization, we hypothesized that maternal exposure to NMs negatively affects male germ cells by altering epigenetic regulation, which may in turn affect embryo development. Small non-coding RNAs (including microRNAs, PIWI-interacting RNAs, tRNA-derived small RNAs, and rRNA-derived small RNAs), which are differentially expressed in mammalian male germ cells in a spatiotemporal manner, could play important regulatory roles in spermatogenesis and embryogenesis. Thus, the evaluation of RNAs responsible for sperm fertility is of great interest in reproductive toxicology and medicine. However, whether the effect of maternal exposure to NMs on spermatogenesis in the offspring (intergenerational effects) really triggers multigenerational effects remains unclear, and infertility biomarkers for evaluating paternal inheritance have not been identified to date. In this review, existing lines of evidence on the effects of prenatal exposure to NMs on male reproduction are summarized. A working hypothesis of the transgenerational effects of sperm-derived epigenomic changes in the F1 generation is presented, in that such maternal exposure could affect early embryonic development followed by deficits in neurodevelopment and male reproduction in the F2 generation.