The Chinese Loess Plateau has undergone extensive revegetation to restore degraded land and enhance carbon sequestration. However, soil organic carbon (SOC) sequestrated in the soil profiles of deep-rooted plants has not been fully studied. Here, we investigated the SOC within a 0–23 m profile in farmlands and apple orchards converted from farmlands with different ages (A5, <5 years; A10, ~10 years; A15, ~15 years; A20, >20 years) and the controlling factors on three loess tablelands (Changwu, Qingyang, and Luochuan). The results show that SOC stocks among farmlands and orchards showed no significant difference (p = 0.88); however, SOC stocks showed a trend with tree ages, i.e., a decrease for A5 and A10 but an increase for A15 and A20. For the vertical variability, the SOC stock was the highest within 0–1 m, regardless of the standing age; however, the SOC stock in this layer only accounted for 8.8% of the total SOC stock (97.93 ± 9.18 kg m−2). Climate accounted for 82% of the variations and controlled the changes in SOC in the 0–1 m range, while soil texture dominated the SOC in the soil below 1 m, accounting for 57% of the SOC variations. The variations in SOC in the thick, unsaturated zones provide implications for future land use management and the sustainability of apple orchards in arid regions.