Environmental factors shape the structure and functioning of benthic communities. In coastal zones of the southwestern Baltic Sea, boulder fields represent one of the most productive habitats, supporting diverse benthic communities that provide many ecosystem services. In this study, the influence of the geological characteristics of boulder fields on the biodiversity of associated hard-bottom communities was investigated at two different spatial scales (few kilometers and tens of kilometers). The analyses on overall richness (taxonomic and functional) and community composition revealed how: (i) locally the size of boulders and (ii) regionally site-specific factors like the boulder density distribution and the sediment distribution can act as environmental driving forces. The overall richness of assemblages was shown to increase with increasing surface area of boulders, by up to 60% for species and up to 40% for functional richness. At both investigated scales, differences in compositional variability (β diversity) of the communities were detected. Locally, smallest boulders hosted more variable communities (β diversity up to 2 times higher), while at the regional level, indications of a larger habitat heterogeneity featuring the highest β diversity were observed. This study exemplifies how geological habitat characteristics shape the biodiversity of boulder field communities. The obtained information could be considered in assessment strategies, in order to avoid misclassifications of habitats naturally limited in biodiversity, making a step forward to the desired objective of protecting, conserving, and managing boulder field communities in the study area and at other comparable sites.