The detection and attribution of vegetation dynamics in drylands is an important step for the development of effective adaptation and mitigation strategies to combat the challenges posed by human activities and climate change. However, due to the spatial heterogeneity and interactive influences of various factors, quantifying the contributions of driving forces on vegetation change remains challenging. In this study, using the normalized difference vegetation index (NDVI) as a proxy of vegetation growth status and coverage, we analyzed the temporal and spatial characteristics of the NDVI in China’s Inner Mongolian grasslands using Theil–Sen slope statistics and Mann–Kendall trend test methods. In addition, using the GeoDetector method, a spatially-based statistical technique, we assessed the individual and interactive influences of natural factors and human activities on vegetation-NDVI change. The results show that the growing season average NDVI exhibited a fluctuating upward trend of 0.003 per year from 2000 to 2018. The areas with significant increases in NDVI (p < 0.05) accounted for 45.63% of the entire region, and they were mainly distributed in the eastern part of the Mu Us sandy land and the eastern areas of the Greater Khingan Range. The regions with a decline in the NDVI were mainly distributed in the central and western regions of the study area. The GeoDetector results revealed that both natural and human factors had significant impacts on changes in the NDVI (p < 0.001). Precipitation, livestock density, wind speed, and population density were the dominant factors affecting NDVI changes in the Inner Mongolian grasslands, explaining more than 15% of the variability, while the contributions of the two topography factors (terrain slope and slope aspect) were relatively low (less than 2%). Furthermore, NDVI changes responded to the changes in the level of specific influencing factors in a nonlinear way, and the interaction of two factors enhanced the effect of each singular factor. The interaction between precipitation and temperature was the highest among all factors, accounting for 39.3% of NDVI variations. Findings from our study may aid policymakers in better understanding the relative importance of various factors and the impacts of the interactions between factors on vegetation change, which has important implications for preventing and mitigating land degradation and achieving sustainable pasture use in dryland ecosystems.