Lipid asymmetry at the plasma membrane is essential for such processes as cell polarity, cytokinesis and phagocytosis1-3. Here we identify the lipid flippase complex, composed of Lem3, Dnf1 or Dnf24, to play a role in the dynamic recycling of the Cdc42 GTPase, a key regulator of cell polarity5, in yeast. By using quantitative microscopy methods, we show that the flippase complex is required for fast dissociation of Cdc42 from the polar cortex by the guanine nucleotide dissociation inhibitor (GDI). A loss of flippase activity, or pharmacological blockage of the inward flipping of phosphatidylethanolamine (PE), a phospholipid with a neutral head group, disrupts Cdc42 polarity maintained by GDI-mediated recycling. PE flipping may reduce the charge interaction between a Cdc42 C-terminal cationic region with the plasma membrane inner leaflet, enriched for the negatively charged lipid phosphatidylserine (PS). Using a reconstituted system with supported lipid bilayers, we show that the relative composition of PE versus PS directly modulates Cdc42 extraction from the membrane by GDI.