Evolution of local adaptation depends critically on the level of gene flow, which, in plants, can be due to either pollen or seed dispersal. Using analytical predictions and individual‐centred simulations, we investigate the specific influence of seed and pollen dispersal on local adaptation in plant populations growing in patchy heterogeneous landscapes. We study the evolution of a polygenic trait subject to stabilizing selection within populations, but divergent selection between populations. Deviations from linkage equilibrium and Hardy–Weinberg equilibrium make different contributions to genotypic variance depending on the dispersal mode. Local genotypic variance, differentiation between populations and genetic load vary with the rate of gene flow but are similar for seed and pollen dispersal, unless the landscape is very heterogeneous. In this case, genetic load is higher in the case of pollen dispersal, which appears to be due to differences in the distribution of genotypic values before selection.