Soil phosphate (Pi) deficiency is major constraint for rice cultivation worldwide. Cellular membranes account for one third of cellular organic phosphorus (P) in the form of phospholipids. Therefore, remobilization of Pi from membrane phospholipids under Pi deficiency can be an important strategy to improve phosphorus use efficiency. Glycerophosphodiester phosphodiesterases (GDPDs) hydrolyse intermediate product of phospholipid catabolism, glycerophosphodiesters to glycerol‐3‐phosphate, a precursor for P and non P‐lipid biosynthesis. Here, we show that OsGDPD2 is a Pi deficiency responsive gene, which is transcriptionally regulated by OsPHR2. In silico analysis of active site residues and enzymatic assays confirmed phosphodiesterase activity of OsGDPD2. All overexpression lines showed higher GDPD activity, Pi content, root growth, and biomass accumulation as compared with wild type. Conversely, silencing of OsGDPD2 led to decreased GDPD activity and Pi content. Notably, most of the P‐containing metabolites and fatty acids were elevated in transgenic lines. Further, quantitative analysis of polar lipids revealed higher accumulation of several classes of phospholipids and galactolipids in overexpression lines indicating a potential role of OsGDPD2 in de novo glycerolipid biosynthesis. Thus, present study provides insights into novel physiological roles of OsGDPD2 in low Pi acclimation in rice.