Early endosomes (EEs) are central hubs for cargo sorting in vesicular trafficking. Cargoes destined for degradation retain in EEs that are eventually delivered to lysosomes, while recycled cargo destined for the plasma membrane (PM) or to the Golgi are segregated into EE buds, a specialized membrane structure transiently generated on EEs during cargo sorting. Until now, the molecular basis of the membrane expansion during the biogenesis of EE buds is completely elusive. Here, we identify a protein complex containing a Vps13 domain-containing lipid transporter SHIP164, ATPase RhoBTB3 and retromer component Vps26B, promotes the membrane expansion in the biogenesis of EE buds at Golgi-EE contacts. SHIP164 depletion specifically reduces the size and number of EE buds marked by Vps26B and actin, and results in less but enlarged EEs, which can be substantially rescued by wild type SHIP164 other than lipid transfer-defective mutants, suggesting a role of lipid transfer of SHIP164 in the process. SHIP164 and RhoBTB3 interact with enzymes for phospholipid synthesis on motile Golgi vesicles, that frequently contact EEs. Functionally, SHIP164 depletion substantially reduces the trafficking of sphingomyelin to the PM, and impairs cell growth. Together, we propose a lipid transport-dependent route from the Golgi to EEs at the contacts required for EE bud biogenesis.