Birch trees are abundant in central and northern Europe and are dominant trees in broadleaved forests. Birches are pioneer trees that produce large quantities of allergenic pollen efficiently dispersed by wind. The pollen load level depends on the sizes and locations of pollen sources, which are important for pollen forecasting models; however, very limited work has been done on this topic in comparison to research on anthropogenic air pollutants. Therefore, we used highly accurate aerial laser scanning (Light Detection and Ranging—LiDAR) data to estimate the size and location of birch pollen sources in 3-dimensional space and to determine their influence on the pollen concentration in Poznań, Poland. LiDAR data were acquired in May 2012. LiDAR point clouds were clipped to birch individuals (mapped in 2012–2014 and in 2019), normalised, filtered, and individual tree crowns higher than 5 m were delineated. Then, the crown surface and volume were calculated and aggregated according to wind direction up to 2 km from the pollen trap. Consistent with LIDAR data, hourly airborne pollen measurements (performed using a Hirst-type, 7-day volumetric trap), wind speed and direction data were obtained in April 2012. We delineated 18,740 birch trees, with an average density of 14.9/0.01 km2, in the study area. The total birch crown surface in the 500–1500 m buffer from the pollen trap was significantly correlated with the pollen concentration aggregated by the wind direction (r = 0.728, p = 0.04). The individual tree crown delineation performed well (r2 ≥ 0.89), but overestimations were observed at high birch densities (> 30 trees/plot). We showed that trees outside forests substantially contribute to the total pollen pool. We suggest that including the vertical dimension and the trees outside the forest in pollen source maps have the potential to improve the quality of pollen forecasting models.