This study presents the first detailed estimate of Rwanda’s nitrogen (N) flows and N footprint for food (NFfood) from 1961 to 2018. Low N fertilizer inputs, substandard production techniques, and inefficient agricultural management practices are focal causes of low crop yields, environmental pollution, and food insecurity. We therefore assessed the N budget, N use efficiency (NUE), virtual N factors (VNFs), soil N mining factors (SNMFs), and N footprint for the agro-food systems of Rwanda with consideration of scenarios of fertilized and unfertilized farms. The total N input to croplands increased from 14.6 kg N ha−1 yr−1 (1960s) to 34.1 kg N ha−1 yr−1 (2010–2018), while the total crop N uptake increased from 18 kg N ha−1yr−1 (1960s) to 28.2 kg N ha−1yr−1 (2010–2018), reflecting a decline of NUE from 124% (1960s) to 85% (2010–2018). Gaseous N losses of NH3, N2O, and NO increased from 0.45 (NH3), 0.03 (N2O), and 0.00 (NO) Gg N yr−1 (1960s) to 6.98 (NH3), 0.58 (N2O), and 0.10 (NO) Gg N yr−1 (2010–2018). Due to the low N inputs, SNMFs were in the range of 0.00 and 2.99 and the rice production, cash-crop production, and livestock production systems have greater SNMFs in Rwanda. The weighted NFfood per capita that presents the actual situation of fertilized and unfertilized croplands increased from 4.0 kg N cap−1 yr−1 (1960s) to 6.3 kg N cap−1 yr−1 (2010–2018). The NFfood per capita would increase from 3.5 kg N cap−1 yr−1 to 4.8 kg N cap−1 yr−1 under a scenario of all croplands without N fertilizer application and increase from 6.0 to 8.7 kg N cap−1 yr−1 under the situation of all croplands receiving N fertilizer. The per capita agro-food production accounted for approximately 58% of the national NFfood. The present study indicates that Rwanda is currently suffering from low N inputs, high soil N depletion, food insecurity, and environmental N losses. Therefore, suggesting that the implementation of N management policies of increasing agricultural N inputs and rehabilitating the degraded soils with organic amendments of human and animal waste needs to be carefully considered in Rwanda.