From bacteria to humans, adaptive immune systems provide learned memories of past infections. Despite their vast biological differences, adaptive immunity shares features from microbes to vertebrates such as emergent immune diversity, long-term coexistence of hosts and pathogens, and fitness pressures from evolving pathogens and adapting hosts, yet there is no conceptual model that addresses all of these together. To address these questions, we propose and solve a simple phenomenological model of CRISPR-based adaptive immunity in microbes. We show that in coexisting phage and bacteria populations, immune diversity in both populations emerges spontaneously and in tandem, that bacteria track phage evolution with a context-dependent lag, and that high levels of diversity are paradoxically linked to low overall CRISPR immunity. We define average immunity, an important summary parameter predicted by our model, and use it to perform synthetic time-shift analyses on available experimental data to reveal different modalities of coevolution. Finally, immune cross-reactivity in our model leads to qualitatively different states of evolutionary dynamics, including an influenza-like traveling wave regime that resembles a similar state in models of vertebrate adaptive immunity. Our results show that CRISPR immunity provides a tractable model, both theoretically and experimentally, to understand general features of adaptive immunity.