Our current understanding of the natural evolution of RNA viruses comes largely from consensus level genetic analyses which ignore the diverse mutant swarms that comprise within-host viral populations. The breadth and composition of viral mutant swarms impact viral fitness and adaptation, and the capacity for swarm plasticity is likely to be particularly important for arthropod-borne viruses (arboviruses) that cycle between taxonomically divergent hosts. Despite this, characterization of the relationship between the selective pressures and genetic signatures of the mutant swarm and consensus sequences is lacking. To clarify this, we analyzed previously generated whole genome, deep-sequencing data from 548 West Nile virus samples isolated from avian tissues or mosquitoes in New York State from 1999–2018. Both consensus level (interhost) and minority level (intrahost) nucleotide and amino acid sequences were analyzed, and diversity at each position was calculated across the genome in order to assess the relationship between minority and consensus sequences for individual genes and hosts. Our results indicate that consensus sequences are an inept representation of the overall genetic diversity. Unique host and gene-specific signatures and selective pressures were identified. These data demonstrate that an accurate and comprehensive understanding of arbovirus evolution and adaptation within and between hosts requires consideration of minority genotypes.