Spatial scale partly explains the differentiated effects of habitat fragmentation on plant biodiversity, but the mechanisms remain unclear. To investigate the effects of habitat fragmentation on genetic diversity at different scales, we sampled Actinidia chinensis Planch. at broad and fine scales, China. The broad-scale sampling included five mountain populations and one oceanic island population (Zhoushan Archipelago), and the fine-scale sampling covered 11 lake islands and three neighboring land populations in Thousand-Island Lake (TIL). These populations were genotyped at 30 microsatellite loci, and genetic diversity, gene flow, and genetic differentiation were evaluated. Genetic differentiation was positively related to geographical distance at the broad scale, indicating an isolation-by-distance effect of habitat fragmentation on genetic diversity. The oceanic population differed from the mainland populations and experienced recent bottleneck events, but it showed high gene flow with low genetic differentiation from a mountain population connected by the Yangtze River. At the fine scale, no negative genetic effects of habitat fragmentation were found because seed dispersal with water facilitates gene flow between islands. The population size of A. chinensis was positively correlated with the area of TIL islands, supporting island biogeography theory, but no correlation was found between genetic diversity and island area. Our results highlight the scale-dependent effects of habitat fragmentation on genetic diversity and the importance of connectivity between island-like isolated habitats at both the broad and fine scales.