Pollinator interactions are important in the evolution of floral traits, given that pollinators can directly influence plant fitness and mating patterns through interactions with flowers. However, geographic variation in both plant traits and floral visitors across multiple populations is largely understudied, despite being ubiquitous. This study explores whether a geographic mosaic of ecological interactions underlies observed patterns of floral divergence 1) among species of the Castilleja purpurea complex (C. purpurea, C. citrina, and C. lindheimeri) and the congener C. sessiliflora, as well as 2) within C. sessiliflora, across its wide geographic range. We sampled floral visitors and floral traits (morphology and color) at 23 populations over 1-3 years, with reproductive fitness (fruit set) data for 18 of these populations. We documented a wide diversity of pollinator functional groups visiting the four focal species, including bees, butterflies, hawkmoths, and hummingbirds. Visitor assemblages varied among species and across geography both in terms of the composition and diversity of floral visitors. We found support for relationships between floral traits and visitation by certain pollinator groups, which often aligned with syndrome-associated predictions. Additionally, we found evidence that visitation from pollinators predicted via syndromes was associated with increased reproductive fitness for two species: the red-flowered C. lindheimeri and its hummingbird visitors, and the long-floral-tubed C. sessiliflora when visited by hawkmoths. Beyond these cases, we found that pollinator functional groups were not restricted to plant species, and instead pollinators were largely generalist in their foraging behavior, suggesting the likelihood for incomplete reproductive isolation and the potential for ongoing gene flow among plant species where their ranges overlap. This study provides a large-scale exploration of how variation in pollinator assemblages across distributions may underlie floral trait divergence within and among recently diverged species, even when characterized by largely generalized modes of pollination. Our extensive sampling of 23 populations over multiple years across a large geographic area highlights the value of range-wide studies for characterizing patterns of divergence and speciation mediated by ecological interactions.