Viruses within a family often vary in their cellular tropism and pathogenicity. In many cases, these variations are due to viruses switching their specificity from one cell surface receptor to another. The structural requirements that underlie such receptor switching are not well understood especially for carbohydrate-binding viruses, as methods capable of structure-specificity studies are only relatively recently being developed for carbohydrates. We have characterized the receptor specificity, structure and infectivity of the human polyomavirus BKPyV, the causative agent of polyomavirus-associated nephropathy, and uncover a molecular switch for binding different carbohydrate receptors. We show that the b-series gangliosides GD3, GD2, GD1b and GT1b all can serve as receptors for BKPyV. The crystal structure of the BKPyV capsid protein VP1 in complex with GD3 reveals contacts with two sialic acid moieties in the receptor, providing a basis for the observed specificity. Comparison with the structure of simian virus 40 (SV40) VP1 bound to ganglioside GM1 identifies the amino acid at position 68 as a determinant of specificity. Mutation of this residue from lysine in BKPyV to serine in SV40 switches the receptor specificity of BKPyV from GD3 to GM1 both in vitro and in cell culture. Our findings highlight the plasticity of viral receptor binding sites and form a template to retarget viruses to different receptors and cell types.