Understanding plant stress memory under extreme temperatures such as cold and heat could contribute to plant development. Plants employ different types of stress memories, such as somatic, intergenerational and transgenerational, regulated by epigenetic changes such as DNA and histone modifications and microRNAs (miRNA), playing a key role in gene regulation from early development to maturity. In most cases, cold and heat stresses result in short-term epigenetic modifications that can return to baseline modification levels after stress cessation. Nevertheless, some of the modifications may be stable and passed on as stress memory, potentially allowing them to be inherited across generations, whereas some of the modifications are reactivated during sexual reproduction or embryogenesis. Several stress-related genes are involved in stress memory inheritance by turning on and off transcription profiles and epigenetic changes. Vernalization is the best example of somatic stress memory. Changes in the chromatin structure of the Flowering Locus C (FLC) gene, a MADS-box transcription factor (TF), maintain cold stress memory during mitosis. FLC expression suppresses flowering at high levels during winter; and during vernalization, B3 TFs, cold memory cis-acting element and polycomb repressive complex 1 and 2 (PRC1 and 2) silence FLC activation. In contrast, the repression of SQUAMOSA promoter-binding protein-like (SPL) TF and the activation of Heat Shock TF (HSFA2) are required for heat stress memory. However, it is still unclear how stress memory is inherited by offspring, and the integrated view of the regulatory mechanisms of stress memory and mitotic and meiotic heritable changes in plants is still scarce. Thus, in this review, we focus on the epigenetic regulation of stress memory and discuss the application of new technologies in developing epigenetic modifications to improve stress memory.