Agrarian expansion and intensification in the tropical agricultural‐forest frontiers (TAFF) are continuously encroaching on forests, yet the magnitudes and processes of agricultural‐forest advances and retreats remain lacking investigation systematically and quantitatively. With over three‐decade (1987–2018) of land‐cover products, here, we revealed the spatiotemporal dynamic processes of agricultural advance and forest retreat and then quantified forest loss (FL) due to cropland and plantation expansion (PE) in Mainland Southeast Asia (MSEA). First, the agricultural expansion and FL peaked in the late 1990s to early 2000s and decelerated afterward in MSEA. Meanwhile, the continuous decline in deforestation rates was accompanied by a decline in the patches of forest fragments, while cropland and plantations first decreased and then increased. Second, 85% of cropland expansion (CE) occurred in forest frontiers of Myanmar (37%) and Thailand (32%) in particular, compared with 72% of PE in plantation‐forest frontiers, for example, 24% in Thailand and 22% in Cambodia. Third, 51% and 37% of the forests that declined were converted into cropland and plantations, respectively, with obvious national variations. In the least developed countries, such as Myanmar and Cambodia, CE dominated (65%), while the proportion of PE in Thailand and Vietnam was up to 56%. Finally, over 10% of the advances and retreats (re‐)occurred in various protected areas, with the expansion ratio of cropland and plantations nearly 1:2, particularly the Cambodia‐Thailand border. We thus appeal for more effective efforts from governments, the scientific community, and international initiatives (e.g., the UN‐REDD and Sustainable Development Goals) to further study and solve the issues of forest retreat and agricultural advance across the entire TAFF. Our new findings and insights about agricultural‐forest advances and retreats in their frontiers can enrich existing pan‐regional research of forest conversion or FL and agricultural expansion and provide an encouraging basis for action to reduce environmental degradation in tropical forests, including protected areas.