Although tailed bacteriophages (phages) of the class Caudoviricetes are thought to constitute the most abundant and ecologically relevant group of phages that can integrate their genome into the host chromosome, it is becoming increasingly clear that other prophages are widespread. Here, we show that prophages derived from filamentous and tailless phages with genome sizes below 16 kb make up the majority of prophages in marine bacteria of the genus Vibrio. To estimate prophage prevalence unaffected by database biases, we combined comparative genomics and chemical induction of 58 diverse Vibrio cyclitrophicus isolates, resulting in 107 well-curated prophages. Complemented with computationally predicted prophages, we obtained 1,158 prophages from 931 naturally co-existing strains of the family Vibrionaceae. Prophages resembling tailless and filamentous phages predominated, accounting for 80% of all prophages in V. cyclitrophicus and 60% across the Vibrionaceae. In our experimental model, prophages of all three viral realms actively replicated upon induction indicating their ability to transfer to new hosts. Indeed, prophages were rapidly gained and lost, as suggested by variable prophage content between closely related V. cyclitrophicus. Prophages related to filamentous and tailless phages were integrated into only three genomic locations and restored the function of their integration site. Despite their small size, they contained highly diverse accessory genes that may contribute to host fitness, such as phage defense systems. We propose that, like their well-studied tailed equivalent, tailless and filamentous temperate phages are active and highly abundant drivers of host ecology and evolution in marine Vibrio, which have been largely overlooked.