Respiratory syncytial virus (RSV) is the leading cause of death from lower respiratory tract infection in infants and children, and is responsible for considerable morbidity and mortality in older adults. Vaccines for pregnant women and elderly which are in phase III clinical studies target people with pre-existing natural immunity against RSV. To investigate the background immunity which will be impacted by vaccination, we single cell-sorted human memory B cells and dissected functional and genetic features of neutralizing antibodies (nAbs) induced by natural infection. Most nAbs recognized both the prefusion and postfusion conformations of the RSV F-protein (cross-binders) while a smaller fraction bound exclusively to the prefusion conformation. Cross-binder nAbs used a wide array of gene rearrangements, while preF-binder nAbs derived mostly from the expansion of B-cell clonotypes from the IGHV1 germline. This latter class of nAbs recognizes an epitope located between Site Ø, Site II, and Site V on the F-protein, identifying an important site of pathogen vulnerability.