Lower nitrogen recovery efficiency (NRE) and negative environmental impacts caused by excessive nitrogen (N) fertilization threaten the sustainability of agriculture. Efficient and appropriate fertilization practices are extremely important to achieve higher crop yield with minimum N loss. A field microplot experiment was conducted in a wheat-maize rotation system in Shaanxi province, at North China Plain, using the 15N isotope tracer technique to qualify the different annual N managements in terms of crop yield, NRE, N distribution in plant-soil, and N losses to optimize the N management. The experiment included four N treatments: conventional practice with 510 kg ha−1 annually in four applications (N1), and three optimized N treatments, reducing N rate to 420 kg ha−1, adjusting topdressing fertilizer times and using slow-release fertilizer (SRF) (N2, N3, N4). The results showed that the grain yield and N uptake did not differ significantly among treatments. N from fertilizer taken up (Ndff) by wheat was not affected by N management; however, in maize, Ndff performed differently. Optimized treatments significantly decreased the Ndff as compared to N1 treatment. Furthermore, NRE of wheat and annual nitrogen recovery efficiency (annual NRE) did not differ among treatments in 2016 but significantly increased in 2017 compared to N1. Annual NRE in 2017 was similar to that obtained for wheat. For maize, optimized N managements decreased the NRE in N3 and N4 treatments of two years. Potential losses in wheat were also similar amongst treatments, but in maize, N3 and N4 had lower residual N in the soil’s top 60 cm but resulted in higher potential losses than N1 and N2. Overall, our results demonstrate that applying 420 kg N ha−1 annually in three applications and combining SRF and urea are effective to sustain crop yield, improve the efficiency of N usage by maize, and reduce N losses in this region.