The carbon balance of the global ecosystems is significantly influenced by the soil organic carbon (SOC) pool in China's terrestrial ecosystems. Nevertheless, the current understanding of the main controlling factors of SOC in different ecosystems and their discrepancies is limited. The goal of this research was to better understand the human and environmental variables affecting SOC in China's terrestrial ecosystems. We designed a conceptual framework using 2674 samples collected from four ecosystems (grasslands, shrublands, wetlands, and croplands) in China during the 2000–2014 period, combining geodetector and multiple regression (MR) approaches to investigate the effects of environmental conditions, human activity, and their interplay on surface SOC (0–20 cm). Results showed that there were large discrepancies in the strength of influencing factors among different ecosystems. Total nitrogen (TN), mean annual temperature (MAT), and bulk density (BD) were the major factors influencing SOC in grasslands. BD, TN, and pH dominated in shrublands. For wetlands, SOC stocks were primarily attributed to maximum temperature (TMMX), MAT, and potential evapotranspiration (PET). Croplands are predominantly controlled by minimum temperature (TMMN), MAT, and TN. These results highlight that natural factors, particularly climatic and soil characteristics, were the dominant factors controlling SOC stocks in China's terrestrial ecosystems. This work also highlights that the interaction of two influencing factors, especially, pairs of soil characteristics factors, pairs of climate and soil characteristics factors, can well explain the drivers of SOC on the surface soil in China. Our study emphasizes the spatial heterogeneity of the factors that influence SOC in terrestrial ecosystems, enhancing knowledge of SOC at the national level, and providing the guideline for devising better policy to improve C sequestration and mitigate climate changes.