SERINC5 is a multi-pass intrinsic membrane protein that suppresses HIV-1 infectivity when incorporated into budding virions. The HIV-1 Nef virulence factor prevents viral incorporation of SERINC5 by triggering its downregulation from the producer cell membrane through an AP-2-dependent endolysosomal pathway. However, the mechanistic basis for SERINC5 downregulation by Nef remains elusive. Here we demonstrate that Nef homodimers are important for SERINC5 downregulation, trafficking to late endosomes, and exclusion from newly synthesized viral particles. Based on previous X-ray crystal structures, we mutated three conserved residues in the Nef dimer interface (L112, Y115, F121) and demonstrated attenuated homodimer formation in a cell-based fluorescence complementation assay. Point mutations at each position reduced the infectivity of HIV-1 produced from transfected 293T cells, the Jurkat TAg T cell line, and donor mononuclear cells in a SERINC5-dependent manner. In SERINC5-transfected 293T cells, virion incorporation of SERINC5 was increased by dimerization-defective Nef mutants, while downregulation of SERINC5 from the membrane of transfected Jurkat cells by these mutants was significantly reduced. Nef dimer interface mutants also failed to trigger internalization of SERINC5 and localization to Rab7+ late endosomes in T cells. Importantly, fluorescence complementation assays demonstrated that dimerization-defective Nef mutants retained interaction with both SERINC5 and AP-2. These results show that downregulation of SERINC5 and subsequent enhancement of viral infectivity requires Nef homodimers and support a mechanism by which the Nef dimer bridges SERINC5 to AP-2 for endocytosis. Pharmacological disruption of Nef homodimers may control HIV-1 infectivity and viral spread by enhancing virion incorporation of SERINC5.