While age-associated B cells (ABCs) are known to expand and persist following viral infection and during autoimmunity, their interactions are yet to be studied together in these contexts. Epstein-Barr virus (EBV) infection has long been implicated in multiple sclerosis (MS), and it is not known whether ABCs could play a role in mediating viral contribution to autoimmunity. Here, we show that the circulating ABC population is expanded in people with MS and that EBV infection and MS status differentially impact the circulating ABC phenotype. We then directly compared ABCs during viral infection and autoimmunity using mouse models of EBV, gammaherpesvirus 68 (γHV68), and MS, experimental autoimmune encephalomyelitis (EAE). We observed that splenic ABCs are expanded in a sex-biased manner during both latent virus infection and EAE, and each event drives the ABC population to opposing phenotypes. We have previously shown that latent γHV68 infection exacerbates EAE and here we show that mice lacking ABCs fail to display γHV68-enhanced disease. Collectively, these findings indicate that latent viral infection and central nervous system autoimmunity differentially impact the ABC population and suggests that viral infections such as EBV prime ABCs to contribute pathogenically in MS.