Genome size can vary widely over relatively short evolutionary time scales and is implicated in form, function and ecological success of a species. Here, we generated 17 new de novo whole genome assemblies and present a holistic view on genome size diversity of the highly diversified, non-model insect order, Trichoptera (caddisflies). We detect large variation in genome size and find strong evidence that transposable element (TE) expansions are the primary driver of genome size evolution: TE expansions contribute to larger genomes in clades with higher ecological diversity and have a major impact on protein-coding gene regions. These TE-gene associations show a linear relationship with increasing genome size. Our findings suggest new hypotheses for future testing, especially the effects of TE activity and TE-gene associations on genome stability, gene expression, phenotypes, and their potential adaptive advantages in groups with high species, ecological, and functional diversities.