Endogenous retroviruses (ERVs) are stably integrated in the genome of vertebrates and inherited as Mendelian genes. The several human ERV (HERV) families and related elements represent up to 5–8% of the DNA of our species. ERVs may be involved in the regulation of adjacent genomic loci, especially promoting the tissue-specific expression of genes; some HERVs may have functional roles, e.g., coding for the placental fusogenic protein, syncytin. This paper reviews the growing evidence about factors that may modulate ERVs, including: cell and tissue types (with special attention to placenta and germ cells), processes related to differentiation and aging, cytokines, agents that disrupt cell functions (e.g., DNA hypomethylating agents) and steroids. Special attention is given to HERVs, due to their possible involvement in autoimmunity and reproduction, as well as altered expression in some cancer types; moreover, different HERV families may deserve specific attention, due to remarkable differences concerning, e.g., expression in tissues. A comparison with factors interacting with murine ERV-related sequences indicates that the mouse may be a useful model for studying some patterns of HERV regulation. Overall, the available evidence identifies the diverse, potential interactions with endogenous or exogenous factors as a promising field for investigating the roles of ERVs in physiology and disease.