DNA can experience “replication stress”, an important source of genome instability, induced by various external or endogenous impediments that slow down or stall DNA synthesis. While genome instability is largely documented to favor both tumor formation and heterogeneity, as well as drug resistance, conversely, excessive instability appears to suppress tumorigenesis and is associated with improved prognosis. These findings support the view that karyotypic diversity, necessary to adapt to selective pressures, may be limited in tumors so as to reduce the risk of excessive instability. This review aims to highlight the contribution of specialized DNA polymerases in limiting extreme genetic instability by allowing DNA replication to occur even in the presence of DNA damage, to either avoid broken forks or favor their repair after collapse. These mechanisms and their key regulators Rad18 and Polθ not only offer diversity and evolutionary advantage by increasing mutagenic events, but also provide cancer cells with a way to escape anti-cancer therapies that target replication forks.