Awareness of the significant benefits of mangroves to human lives and their role in regulating environmental processes has increased during the recent decades. Yet there remains significant uncertainty about the mangrove change trajectories and the drivers of change at national scales. In Colombia, the absence of historical satellite imagery and persistent cloud cover have impeded the accurate mapping of mangrove extent and change over time. We create a temporally consistent Landsat-derived dataset using the LandTrendr algorithm to track the historical land cover and mangrove conversion from 1984-2020 across Colombia. Over this period, mangrove extent decreased by ~48.000ha (14% of total mangrove area). We find a gradual reduction of mangrove extent along the Pacific coast since 2004, whereas, in the Caribbean, mangrove cover declined around during 1984-1988 and also after 2012. Our time-series analysis matches with drivers of mangrove change at three local sites. For instance, hydroclimatic events, dredging activities, and high sediment loads transported by the rivers have collectively improved mangrove recovery in some sites. In contrast, human activities pressure linked to agricultural expansion and road construction have degraded mangroves. The transition from dense mangrove to other vegetation types is the most significant conversion affecting mangrove cover in Colombia, impacting an area of 38,469 ± 2,829 ha. We anticipate increased mangrove loss, especially along the Pacific coast, resulting from intensified human activity. Prioritization of conservation areas is needed to support local institutions, maintain currently protected areas, and develop strategies (e.g. payment for ecosystem services) to preserve one of the most pristine mangrove regions in the Western Hemisphere.