Wild bumblebees are key pollinators of crops and wild plants that rely on the continuous availability of floral resources. A better understanding of the spatio-temporal availability and use of floral food resources may help to promote bumblebees and their pollination services in agricultural landscapes. We placed colonies of Bombus terrestris L. in 24 agricultural landscapes with various degrees of floral resource availability and assessed different parameters of colony growth and fitness. We estimated pollen availability during different periods of colony development based on detailed information of the bumblebee pollen diet and the spatial distribution of the visited plant species. Total pollen availability did not significantly explain colony growth or fitness. However, when using habitat maps, the weight gain of colonies, the number of queen cells, and colony survival decreased with increasing distance from the forest. The better explanation of bumblebee performance by forest proximity than by (plant-inferred) pollen availability indicates that other functions of forests than pollen provision were important. The conservation of forests next to agricultural land might help to sustain high populations of these important wild pollinators and enhance their crop pollination services. Combining different mapping approaches might help to further disentangle complex relationships between B. terrestris and their environment in agricultural landscapes.