SARS-CoV-2 has caused a global pandemic that has infected more than 230 million people worldwide. Although several vaccine candidates have received emergency use authorization (EUA), there is still limited knowledge on how vaccine dosing affects immune responses. We performed mechanistic studies in mice to understand how the priming dose of an adenovirus-based SARS-CoV-2 vaccine affects long-term immunity to SARS-CoV-2. We first primed C57BL/6 mice with an adenovirus serotype 5 vaccine encoding the SARS-CoV-2 spike protein, similar to that used in the CanSino and Sputnik V vaccines. The vaccine prime was administered either at a standard dose or 1000-fold lower dose, followed by a boost with the standard dose four weeks later. Initially, the low dose prime induced lower immune responses relative to the standard dose prime. However, the low dose prime elicited immune responses that were qualitatively superior, and upon boosting, exhibited significantly more potent recall and functional capacity. We also report similar effects with a simian immunodeficiency virus (SIV) vaccine. These findings show an unexpected advantage of fractionating vaccine prime doses, warranting a re-evaluation of vaccine trial protocols for SARS-CoV-2 and other pathogens.