Compared to conventional approaches, the integration of population size analysis with habitat suitability assessment on a large scale can provide more evidence to explain the mechanisms of habitat isolation and fragmentation, and thus make regional conservation plans. In this paper, we analyzed the habitat suitability for giant pandas in the Minshan Mountains, China, using the ecological-niche factor analysis (ENFA) method, and then evaluated the current conservation status of this endangered species. The results showed that (1) giant pandas were distributed in a narrow altitudinal range in which vegetation cover was dominated by coniferous forest, mixed coniferous and deciduous broadleaf forest, and deciduous broadleaf forest with scattered bamboo understory, and (2) roads and human settlements had strong negative effects on the panda habitat selection. According to habitat analysis, the total habitat area of giant panda in the Minshan Mountains was 953,173 ha, which was fragmented into 12 habitat units by major roads, rivers, and human settlements. The habitat of the mid-Minshan was less fragmentized, but was seriously fragmented in the north. The panda population size estimation showed that 676 individuals inhabited the study area, and 94.53% of them were in the mid-Minshan, but small panda populations less than 30 individuals inhabited the isolated and fragmented habitat patches in the north. The nature reserves in the Minshan Mountains have formed three conservation groups, which covered 41.26% of panda habitat and protected 70.71% of panda population of the study area, but there still exists two conservation gaps, and the connectivity among these reserves is still weak. Due to habitat isolation and extensive human disturbances, giant pandas in the north (i.e., Diebu, Zhouqu, and Wudou) are facing threats of local extinction. In order to protect pandas and their habitats in this area, some effective conservation approaches, such as establishing new reserves in gap areas, creating corridors among patches, and seasonally controlling traffic flux in key roads, should be implemented in the future to link these isolated habitats together.