The island rule, a pattern of size shifts on islands, is an oft-cited but little understood phenomenon of evolutionary biology. Here, we explore the evolutionary mechanisms behind the rule in 184 mammal species, testing climatic, ecological and phylogenetic hypotheses in a robust quantitative framework. Our findings confirm the importance of species' ecological traits in determining both the strength and the direction of body size changes on islands. Although the island rule pattern appears relatively weak overall, we find strongest support for models incorporating trait, climatic and geographical factors in a phylogenetic context, lending support to the idea that the island rule is a complex phenomenon driven by interacting intrinsic and extrinsic mechanisms. Overall, we find that different clades may be evolutionarily predisposed to dwarfism or gigantism, but the magnitude of size changes depends more on adaptation to the novel island environment.