Rice cultivation usually involves high water and fertilizer application rates leading to the nonpoint pollution of surface waters with phosphorus (P) and nitrogen (N). Here, a 10-year field experiment was conducted to investigate N and P losses and their impact factors under different irrigation and fertilization regimes. Results indicated that T2 (Chemical fertilizer of 240 kg N ha−1, 52 kg P ha−1, and 198 kg K ha−1 combined with shallow intermittent irrigation) decreased N loss by 48.9% compared with T1 (Chemical fertilizer of 273 kg N ha−1, 59 kg P ha−1, and 112 kg K ha−1 combined with traditional flooding irrigation). The loss ratio (total N loss loading/amount of applied N) of N was 9.24–15.90%, whereas that of P was 1.13–1.31% in all treatments. Nitrate N (NO3-−N) loss was the major proportion accounting for 88.30–90.65% of dissolved inorganic N loss through surface runoff. Moreover, the N runoff loss was mainly due to high fertilizer input, soil NO3-−N, and ammonium N (NH4+−N) contents. In addition, the N loss was accelerated by Bacteroidetes, Proteobacteria, Planotomycetes, Nitrospirae, Firmicutes bacteria and Ascomycota fungi, but decreased by Chytridiomycota fungi whose contribution to the N transformation process. Furthermore, T2 increased agronomic N use efficiency (AEN) and rice yield by 32.81% and 7.36%, respectively, in comparison with T1. These findings demonstrated that T2 might be an effective approach to ameliorate soil chemical properties, regulate microbial community structure, increase AEN and consequently reduce N losses as well as maintaining rice yields in the present study.