Mangrove wetlands are hotspots of global biodiversity and blue carbon reserves in coastal wetlands, with unique ecological functions and significant socioeconomic value. Annual fine-scale monitoring of mangroves is crucial for evaluating national conservation programs and implementing sustainable mangrove management strategies. However, annual fine-scale mapping of mangroves over large areas using remote sensing remains a challenge due to spectral similarities with coastal vegetation, tidal periodic fluctuations, and the need for consistent and dependable samples across different years. In previous research, there has been a lack of strategies that simultaneously consider spatial, temporal, and methodological aspects of mangrove extraction. Therefore, based on an approach that considers mangrove habitat, tides, and a semantic segmentation approach, we propose a method for fine-scale mangrove mapping suitable for long time-series data. This is an optimized hybrid model that integrates spatial, temporal, and methodological considerations. The model uses five sensors (GF-1, GF-2, GF-6, ZY-301, ZY-302) to combine deep learning U-Net models with mangrove habitat information and algorithms during low-tide periods. This method produces a mangrove map with a spatial resolution of 2 m. We applied this algorithm to three typical mangrove regions in the Beibu Gulf of Guangxi Province. The results showed the following: (1) The model scored above 0.9 in terms of its F1-score in all three study areas at the time of training, with an average accuracy of 92.54% for mangrove extraction. (2) The average overall accuracy (OA) for the extraction of mangrove distribution in three typical areas in the Beibu Gulf was 93.29%. When comparing the validation of different regions and years, the overall OA accuracy exceeded 89.84% and the Kappa coefficient exceeded 0.74. (3) The model results are reliable for extracting sparse and slow-growing young mangroves and narrow mangrove belts along roadsides. In some areas where tidal flooding occurs, the existing dataset underestimates mangrove extraction to a certain extent. The fine-scale mangrove extraction method provides a foundation for the implementation of fine-scale management of mangrove ecosystems, support for species diversity conservation, blue carbon recovery, and sustainable development goals related to coastal development.