Nitrogen-use efficiency (NUE) of crops is critical for addressing global food security in response to climate change effects. The objective of our study was to evaluate the effects of N fertilization integrated with crop residue management to optimize NUE of crops. A replicated 2 x 4 split-plot experiment in randomized complete block design was conducted during the 2020–2022 on farmers’ fields at Bogura, Cumilla, and Mymensingh sites of Bangladesh. The main plots were two levels of crop residues (no residue, NR vs. 30% residue, CR) combined with sub-plots of four levels of N fertilizers (control, CL; farmers’ practice, FP; national recommended dose, RD, and 125% of RD, 125RD) on potato (Solanum tuberosum, L.), maize (Zea mays, L.), and rice (Oryza sativa, L.) systems. N fertilization when interacted with residue retention progressively increased crop yield. Residue alone contributed significantly to increase NUE of potato and maize by 9 and 6%, respectively. The NUE of rice increased significantly, while the NUE of maize decreased with N fertilization. The comparison of NUE models suggested that the nationally recommended N fertilization should be adjusted to 169, 152, 187, 92, and 112 kg/ha for potato, maize, Boro, Aus, and Aman rice, respectively. For potato and Aman rice, the SNMI showed better results, whereas for Boro, Aus, and maize, utilizing both NUE and SNMI provided maximum benefit from N management.