Most known SARS-CoV-2 neutralizing antibodies (nAbs), including those approved by the FDA for emergency use, inhibit viral infection by targeting the receptor-binding domain (RBD) of the spike (S) protein. Variants of concern (VOC) carrying mutations in the RBD or other regions of S reduce the effectiveness of many nAbs and vaccines by evading neutralization. Therefore, therapies that are less susceptible to resistance are urgently needed. Here, we characterized the memory B-cell repertoire of COVID-19 convalescent donors and analyzed their RBD and non-RBD nAbs. We found that many of the non-RBD-targeting nAbs were specific to the N-terminal domain (NTD). Using neutralization assays with authentic SARS-CoV-2 and a recombinant vesicular stomatitis virus carrying SARS-CoV-2 S protein (rVSV-SARS2), we defined a panel of potent RBD and NTD nAbs. Next, we used a combination of neutralization-escape rVSV-SARS2 mutants and a yeast display library of RBD mutants to map their epitopes. The most potent RBD nAb competed with hACE2 binding and targeted an epitope that includes residue F490. The most potent NTD nAb epitope included Y145, K150 and W152. As seen with some of the natural VOC, the neutralization potencies of COVID-19 convalescent sera were reduced by 4-16-fold against rVSV-SARS2 bearing Y145D, K150E or W152R spike mutations. Moreover, we found that combining RBD and NTD nAbs modestly enhanced their neutralization potential. Notably, the same combination of RBD and NTD nAbs limited the development of neutralization-escape mutants in vitro, suggesting such a strategy may have higher efficacy and utility for mitigating the emergence of VOC.