The Sloping Land Conversion Program (SLCP) is a significant measure to achieve the Sustainable Development Goals (SDGs) proposed by the United Nations in 2015. SLCP plays an important role in poverty alleviation and income increase for farmers in poor areas. The purpose of this study is to analyze whether the income of farmers has increased after participating in SLCP, and whether SLCP has released the agricultural labor force to obtain non-agricultural income by participating in non-agricultural work. Based on the field investigation in Luocheng County and Longsheng County of Guangxi, Libo County, and Dushan county of Guizhou, this paper uses the method of propensity score matching (PSM) to explore the impact of SLCP on the income of farmers in rocky desertification areas. According to our research, it is found that: (1) SLCP has a positive effect of 5.2% on the average annual net income of farmers, a positive effect of 43.2% on agricultural income, and a negative effect of 9.8% on non-agricultural income, but all of the effects are insignificant. Selective deviation will overestimate the impact of SLCP on farmers’ total income and agricultural income and underestimate the impact on non-agricultural income. SLCP failed to promote the transformation of farmers into secondary and tertiary industries. The mechanism of SLCP to increase farmers’ income is complex. (2) Farmers’ participation in SLCP is influenced by work experience and education level in human capital, participation in skills training in social capital, and owning durable consumer goods in physical capital. Although SLCP will promote economic development under the condition of improving the ecological environment in the future, it is not advisable to exchange farmers’ livelihood for ecological construction at present. The implementation of SLCP should consider not only the overall ecological benefits, but also the short-term social and economic benefits.