In the context of economic boom and climate change, monitoring the spatio-temporal dynamics of forest fragmentation induced by disturbances and understanding its corresponding associated factors are critical for developing informed forest management strategies. In this study, based on multi-temporal Landsat images acquired from 1999 to 2020, a SVM classifier was first applied to produce high-accuracy land cover maps in Xinyu City. Next, morphological spatial pattern analysis (MSPA) was implemented to characterize the spatio-temporal patterns of forest fragmentation by producing maps of seven fragmentation components, including the core, islet, perforation, edge, bridge, loop, and branch. Then, both natural and human factors responsible for the observed forest fragmentation dynamics were analyzed using the geo-detector model (GDM). The results showed that over the past two decades, Xinyu City experienced a process of significant forest area loss and exacerbating forest fragmentation. The forest area decreased from 1597.35 km2 in 1999 to 1372.05 km2 in 2020. The areal ratio of core patches decreased by 8.49%, and the areal ratio of edge patches increased by 5.98%. Spatially, the trend of forest fragmentation exhibited a progressive increase from the southern and northern regions towards the central and eastern areas. Large-scale forest core patches were primarily concentrated in the northwestern and southwestern regions, while smaller core patches were found in the eastern and central areas. Notably, human activities, such as distance from the roads and land use diversity, were identified as significantly associated with forest fragmentation. The interaction effect of these factors had a greater impact on forest fragmentation than their individual contributions. In conclusion, Xinyu City possesses the potential to further alleviate forest fragmentation by employing the regional differentiation development strategies: (1) intensive development in the northwest and southern regions; (2) high-density development in the western, northwestern, and southern regions, and (3) conservation development in the southwest, northeast, and east-central regions, thus aligning with the path of local social advancement.