CRISPR-Cas systems provide bacteria with both specificity and adaptability in defence against invading genetic elements. From a theoretical perspective, CRISPR-Cas systems confer many benefits. However, they are observed at an unexpectedly low prevalence across the bacterial domain. While these defence systems can be gained horizontally, fitness costs may lead to selection against their carriage. Understanding the source of CRISPR-related fitness costs will help us to understand the evolutionary dynamics of CRISPR-Cas systems and their role in shaping bacterial genome evolution. Here, we review our current understanding of the potential fitness costs associated with CRISPR-Cas systems. In addition to potentially restricting the acquisition of genetic material that could confer fitness benefits, we explore five alternative biological factors that from a theoretical perspective may influence the fitness costs associated with CRISPR-Cas system carriage: (1) the repertoire of defence mechanisms a bacterium has available to it, (2) the potential for a metabolic burden, (3) larger-scale population and environmental factors, (4) the phenomenon of self-targeting spacers, and (5) alternative non-defence roles for CRISPR-Cas.