The factors shaping microbial communities in marine subsurface sediments remain poorly understood. Here, we analyzed the microbiome of subsurface sediments within a depth range of 1.6–1.9 m, at 10 locations along the Oregon coast. We used metagenomics to reconstruct the functional structure and 16S rRNA gene amplicon sequencing to estimate the taxonomic composition of microbial communities, accompanied by physicochemical measurements. Functional community structure, in terms of the proportions of various gene groups, was remarkably stable across samples, despite the latter covering a region spanning over 300 km. In contrast, taxonomic composition was highly variable, especially at the level of amplicon sequence variants (ASVs) and operational taxonomic units (OTUs). Mantel correlation tests between compositional dissimilarities and geographic distances revealed only a moderate influence of distance on composition. Regression models predicting taxonomic dissimilarities and considering up to 20 physicochemical variables as predictors, almost always failed to select a significant predictor, suggesting that variation in local conditions does not explain the high taxonomic variability. Permutation null models of community assembly revealed that taxa tend to strongly segregate, i.e., exclude each other. We conclude that biological interactions are important drivers of taxonomic variation in subsurface sediments, and that this variation can decouple from functional structure.