Virology has largely focused on viruses that are pathogenic to humans or to the other species that we care most about. There is no doubt that this has been a worthwhile investment. But many transformative advances have been made through the in-depth study of relatively obscure viruses that do not appear on lists of prioritized pathogens. In this review, I highlight the benefits that can accrue from the study of viruses and hosts off the beaten track. I take stock of viral sequence diversity across host taxa as an estimate of the bias that exists in our understanding of host-virus interactions. I describe the gains that have been made through the metagenomic discovery of thousands of new viruses in previously unsampled hosts as well as the limitations of metagenomic surveys. I conclude by suggesting that the study of viruses that naturally infect existing and emerging model organisms represents an opportunity to push virology forward in useful and hard to predict ways. Expected final online publication date for the Annual Review of Virology, Volume 9 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.