The recently emerged SARS-CoV-2 virus has led to a prolonged pandemic characterized by ongoing viral evolution. Vaccines have been an important piece in the strategy to combat the virus but have been insufficient to contain it as the virus continues to evolve to evade immunity developed by vaccination and infection. A consistent argument is that vaccination or prior immunity will lead to less severe infections. In this review, we address the question of whether the virus can evolve to become more virulent, despite prior infection. We describe the intrinsic characteristics of the virus and their relationship to altered virulence. We show that it is likely that viral evolution is subject to evolutionary drift, and it cannot be assumed that the virus will necessarily evolve to be less virulent, or that prior immunity will offer durable protection against severe disease. This has strong implications for public health strategies to confront the ongoing challenges presented by SARS-CoV-2 and implies that there are significant risks to a strategy based on the assumption of waning virulence.