Plants influence belowground microbial communities, which in turn drive nutrient cycling, plant productivity, and the maintenance of plant diversity. Over time, associations between plant communities and their belowground microbiota may strengthen and become more specific. However, we do not know whether plant diversity affects bulk soil microbial community composition and diversity, and whether this association strengthens as micro-evolutionary processes modify plant communities through time. To address this, we used plant communities and soil from an eight-year-old biodiversity experiment (the Jena Experiment; ″old″ plant communities and ″old″ soil), plant communities without such history (″new″ plants), and soil from which any soil legacy was removed by sterilization (″new″ soil) to set up a field biodiversity experiment in which we factorially crossed plant community age with soil legacy for 52 different plant species compositions. We grew these plant-soil communities for four years in experimental plots with plant species richness ranging from 1, 2, 4, 8 to 60 species. At the end of the experiment, we sequenced bacterial 16S rRNA and fungal ITS fragments to compare soil microbial communities under new vs. old plant communities in new vs. old soils. Soil microbial diversity was positively but weakly associated with plant diversity, with the strongest difference in microbial diversity between plant monocultures and any community of more than one plant species. Particular plant community compositions were associated with particular microbial community compositions. Soil legacy (old soil) increased soil bacterial richness and evenness more strongly than did plant community age. Plant community age (old plant communities) significantly affected the abundance of 10% of fungal OTUs. Some of the effects of soil legacy on bacterial and fungal diversity were indirect via changes in soil abiotic and biotic properties. Our findings suggest that as experimental ecosystems develop over time, plant communities associate with specific microbiomes and plant species associate with specific soil microbial species. With increasing knowledge about the taxonomic identities of the microbial OTUs, our dataset in the future may be used for function-based analyses of the detected associations. This may eventually help to identify microbiomes that improve plant health and consequently plant community productivity.