The evolution of floral traits in animal-pollinated plants involves the interaction between flowers as signal senders and pollinators as signal receivers. Flower colors are very diverse, effect pollinator attraction and flower foraging behavior, and are hypothesized to be shaped through pollinator-mediated selection. However, most of our current understanding of flower color evolution arises from variation between discrete color morphs and completed color shifts accompanying pollinator shifts, while evidence for pollinator-mediated selection on continuous variation in flower colors within populations is still scarce. In this review, we summarize experiments quantifying selection on continuous flower color variation in natural plant populations in the context of pollinator interactions. We found that evidence for significant pollinator-mediated selection is surprisingly limited among existing studies. We propose several possible explanations related to the complexity in the interaction between the colors of flowers and the sensory and cognitive abilities of pollinators as well as pollinator behavioral responses, on the one hand, and the distribution of variation in color phenotypes and fitness, on the other hand. We emphasize currently persisting weaknesses in experimental procedures, and provide some suggestions for how to improve methodology. In conclusion, we encourage future research to bring together plant and animal scientists to jointly forward our understanding of the mechanisms and circumstances of pollinator-mediated selection on flower color.