Following land degradation and the associated reduction in crop production, smallholder farmers in the highlands of northwestern Ethiopia have recently practiced Acacia decurrens‐based fallow system. Despite the rapid expansion of this new cropping system, its effect on soil properties and crop productivity has never been exhaustively studied. We assessed the effects of A. decurrens‐based fallow system on soil properties and tef (Eragrostis tef) productivity under the existing production environment on farmers' fields using: (i) tef cultivation under the fallow system in two rotations which represent 5 and 10 years of continuous fallow period (fallowed lands); (ii) conventional tef cultivation system (ST system); (iii) tef cultivation on fallowed lands in the first rotation for two consecutive years; (iv) ST system adjoining a 4‐year‐old acacia woodlot; and (v) interview with local farmers. The results showed that fallowed lands in the first and second rotations improved soil properties and tef productivity with greater effects in the second rotation. Compared to the ST system, fallowed lands in both rotations significantly increased soil pH, soil organic carbon (SOC), available P, total N and cation exchange capacity (CEC), but decreased exchangeable Al3+. Tef aboveground biomass and grain yields were increased by 74% and 93% in the first rotation and 103% and 167% in the second rotation over the ST system, respectively. Similarly, height was increased by 9% in the first rotation and 17% in the second rotation. Fallowed lands sustained positive effects on soil fertility and tef productivity over two successive production years highlighting that the fallow system can benefit farmers through the cultivation of two annual crops in 2 years. Significant tef yield reductions occurred over the first 10 m distance from the acacia woodlot. Soil moisture, pH, SOC, available P, total N and CEC were higher at croplands close to the acacia woodlot. According to the farmers, the fallow system can improve tef productivity and reduce the need for inorganic fertilizer by 50%. Overall, the results demonstrated that fallowing with A. decurrens can reduce soil acidity and restore nutrients for sustainable crop production; and the shading effect could be the limiting factor for tef production on croplands adjacent to the acacia woodlots.