Soil microbiota plays a significant role in plant development and health and appears to be a major component of certain forms of grapevine decline. A greenhouse experiment was conducted to study the impact of the microbiological quality of the soil and grapevine rootstock genotype on the root microbial community and development of young plants. Two rootstocks heterografted with the same scion were grown in two vineyard soils differing in microbial composition and activities. After 4 months, culture-dependent approaches and amplicon sequencing of bacterial 16S rRNA gene and fungal ITS were performed on roots, rhizosphere and bulk soil samples. The root mycorrhizal colonization and number of cultivable microorganisms in the rhizosphere compartment of both genotypes were clearly influenced by the soil status. The fungal diversity and richness were dependent on the soil status and the rootstock, whereas bacterial richness was affected by the genotype only. Fungal genera associated with grapevine diseases were more abundant in declining soil and related root samples. The rootstock affected the compartmentalization of microbial communities, underscoring its influence on microorganism selection. Fluorescence in situ hybridization (FISH) confirmed the presence of predominant root-associated bacteria. These results emphasized the importance of rootstock genotype and soil composition in shaping the microbiome of young vines.