Evidence documenting the decline of insect populations is accumulating. Efforts have increased to mitigate pollinator losses by establishing gardens to support pollinator diversity. However, knowledge of the specific garden characteristics, landscape features, and environmental factors that affect pollinator diversity and abundance is limited, particularly in biodiverse regions in North America. In order to better understand how garden characteristics affect pollinator attraction, we compared pollinator composition across 16 pollinator gardens in the Appalachian ecoregion in North America. We evaluated the effects of garden characteristics (e.g., plant richness, flower abundance, garden size, and proportion of native species), landscape features (land‐use type and distance to forest), and environmental factors (sunlight exposure) on pollinator richness, overall visitation rate, and visitation rate by defined pollinator groups (i.e., solitary native bees, bumblebees, honeybees, lepidopterans, and other insects). Solitary bees (i.e., native bees besides Bombus) were the most frequent visitors (61%). We found differences in pollinator species composition between urban and rural gardens. Moreover, plant richness had a positive effect on pollinator richness and an increase in flower abundance increased the pollinator visitation rate. Flower abundance, plant richness, and high sunlight exposure increased visitation rate of solitary bees. Visitation rate of solitary bees however decreased with increasing proportion of native plants. Overall, our results indicate that garden characteristics, landscape features, and environmental factors all are important mediators of pollinator diversity and abundance. Solitary bees were most affected by garden (i.e., plant richness, number of flowers, and proportion of native plants) and environmental factors (i.e., sunlight exposure). However, we also identified differential effects of garden and environmental factors across pollinator groups. We suggest that an integrated management approach that considers multiple garden and environmental characteristics could help improve the effectiveness of pollinator garden as a conservation tool and help preserve this key ecosystem service.