Desertification of sandy grasslands is an increasing problem, with serious negative impacts on ecosystem functions. Sandy grasslands are fragile with low ecosystem productivity mainly because of the sandy soil structure with low water and nutrient holding capacities and especially low levels of nitrogen. Here, we evaluate the long‐term impacts of sediment addition from a local reservoir, and grass and legume cultivation (artificial grasslands) on a sandy grassland in eastern Inner Mongolia, China. The results showed that even after 32 years, sediment addition had improved soil structure significantly, that is, increasing of silt and clay contents, soil bulk density, and water holding capacity. As the result of improved soil structure, ecosystem functions, including aboveground net primary productivity (ANPP) and soil carbon, nitrogen (N), and phosphorus storage, increased significantly. Net C, N, and P sequestration increased even after accounting for the sediment addition, due, at least partially, to the greater plant biomass trapping large quantities of wind‐blown dust. Plant cultivation, especially the addition of a legume, further increased ANPP significantly, that is, the cultivation of Leymus chinensis and the legume Medicago sativa increased ANPP 6.99 and 44.62 times, respectively. Our study highlights that improvements in soil structure and cultivation with legume species can increased substantially the productivity of sandy grasslands and that the initial increases in grass biomass promoted the sequestration of wind‐blown dust, which helped sustain the increases in productivity.