The Loess Plateau exemplifies the type of ecologically fragile region that faces severe poverty challenges in China. Orchards have expanded rapidly over the past few decades and now constitute a considerable part of local economy. Not only do the characteristics of orchard expansion affect local economic development, but also exert additional pressure on the ecological environment. Therefore, it is essential for sustainable development on the Loess Plateau to investigate the characteristics and driving forces of orchard expansion. The Fuxian, Luochuan, Huangling, three typical orchard planting counties were chosen as the study area. Firstly, the orchard was extracted from the land use/cover classification from 1990–2020. It broadens the research approach to the identification of expansion cash crops by using the combination of linear spectral mixture analysis (LSMA) and decision tree. Secondly, the spatiotemporal dynamics of orchard expansion were quantitatively investigated based on spatial geometry center shift, physical geographical features, landscape pattern and orchard planting suitability. Then, we constructed an evaluation indicators system to detect the feature importance and partial dependence of different factors by random forest regression. It is more innovative to employ the machine learning method to investigate driving forces. Finally, the linkages between planting suitability and orchard expansion were further discussed, and subsequent policies were proposed. Findings demonstrated the orchard had continuously expanded over the past 30 years, with the fastest expansion rate during 1990–2005. Increased cohesion was accompanied by a shift in the orchard’s spatial distribution to the north central region and highly suitable planting regions. Slope turned out to be the primary factor affecting the orchard expansion. In the future, regions with aging orchard but high planting suitability should be the preferred choice for orchard expansion. Additionally, the transportation connectivity and governmental assistance are crucial considerations for the future planning of the orchard.