Upon phosphorus (P) deficiency, marine phytoplankton reduce their requirements for P by replacing membrane phospholipids with alternative non-phosphorus lipids. It was very recently demonstrated that a SAR11 isolate also shares this capability when phosphate starved in culture. Yet, the extent to which this process occurs in other marine heterotrophic bacteria and in the natural environment is unknown. Here, we demonstrate that the substitution of membrane phospholipids for a variety of non-phosphorus lipids is a conserved response to P deficiency among phylogenetically diverse marine heterotrophic bacteria, including members of the Alphaproteobacteria and Flavobacteria. By deletion mutagenesis and complementation in the model marine bacterium Phaeobacter sp. MED193 and heterologous expression in recombinant Escherichia coli, we confirm the roles of a phospholipase C (PlcP) and a glycosyltransferase in lipid remodelling. Analyses of the Global Ocean Sampling and Tara Oceans metagenome data sets demonstrate that PlcP is particularly abundant in areas characterized by low phosphate concentrations. Furthermore, we show that lipid remodelling occurs seasonally and responds to changing nutrient conditions in natural microbial communities from the Mediterranean Sea. Together, our results point to the key role of lipid substitution as an adaptive strategy enabling heterotrophic bacteria to thrive in the vast P-depleted areas of the ocean.