Inorganic ammonium ions are assimilated by a coupled reaction of glutamine synthetase and glutamate synthase (GOGAT). In rice, three genes encoding either ferredoxin (Fd)-GOGAT, NADH-GOGAT1, or NADH-GOGAT2, have been identified. OsNADH-GOGAT2, a newly identified gene, was expressed mainly in fully expanded leaf blades and leaf sheaths. Although the distinct expression profile to OsNADH-GOGAT1, which is mainly detected in root tips, developing leaf blades, and grains, was shown in our previous studies, physiological role of NADH-GOGAT2 is not yet known. Here, we isolated retrotransposon mediated-knockout mutants lacking OsNADH-GOGAT2. In rice grown under paddy field conditions, disruption of the OsNADH-GOGAT2 gene caused a remarkable decrease in spikelet number per panicle associated with a reductions in yield and whole plant biomass, when compared with wild-type (WT) plants. The total nitrogen contents in the senescing leaf blade of the mutants were approximately a half of the WT plants. Expression of this gene was mainly detected in phloem companion cells and phloem parenchyma cells associated with large vascular bundles in fully expanded leaf blades, when the promoter region fused with a β-glucuronidase gene was introduced into the WT rice. These results suggest that the NADH-GOGAT2 is important in the process of glutamine generation in senescing leaves for the remobilization of leaf nitrogen through phloem to the panicle during natural senescence. These results also indicate that other GOGATs, i.e., NADH-GOGAT1 and ferredoxin-GOGAT are not able to compensate the function of NADH-GOGAT2.