Bacteriophages and plasmids can introduce novel DNA into bacterial cells, thereby creating an opportunity for genome expansion; conversely, CRISPR, the prokaryotic adaptive immune system, which targets and eliminates foreign DNAs, may impair genome expansions. Recent studies presented conflicting results over the impact of CRISPR on genome expansion. In this study, we assembled a comprehensive dataset of prokaryotic genomes and identified their associations with phages and plasmids. We found that genomes associated with phages and/or plasmids were significantly larger than those without, indicating that both phages and plasmids contribute to genome expansion. Genomes were increasingly larger with increasing numbers of associated phages or plasmids.Conversely, genomes with CRISPR systems were significantly smaller than those without, indicating that CRISPR has a negative impact on genome size. These results confirmed that on evolutionary timescales, bacteriophages and plasmids facilitate genome expansion, while CRISPR impairs such a process in prokaryotes. Furthermore, our results also revealed that CRISPR systems show a strong preference for targeting phages over plasmids.