Vascular pathogens are the causal agents of some of the most devastating plant diseases in the world, which can cause, under specific conditions, the destruction of entire crops. These plant pathogens activate a range of physiological and immune reactions in the host plant following infection, which may trigger the proliferation of a specific microbiome to combat them by, among others, inhibiting their growth and/or competing for space. Nowadays, it has been demonstrated that the plant microbiome can be modified by transplanting specific members of the microbiome, with exciting results for the control of plant diseases. However, its practical application in agriculture for the control of vascular plant pathogens is hampered by the limited knowledge of the plant endosphere, and, in particular, of the xylem niche. In this review, we present a comprehensive overview of how research on the plant microbiome has evolved during the last decades to unravel the factors and complex interactions that affect the associated microbial communities and their surrounding environment, focusing on the microbial communities inhabiting the xylem vessels of olive trees (Olea europaea subsp. europaea), the most ancient and important woody crop in the Mediterranean Basin. For that purpose, we have highlighted the role of xylem composition and its associated microorganisms in plants by describing the methodological approaches explored to study xylem microbiota, starting from the methods used to extract xylem microbial communities to their assessment by culture-dependent and next-generation sequencing approaches. Additionally, we have categorized some of the key biotic and abiotic factors, such as the host plant niche and genotype, the environment and the infection with vascular pathogens, that can be potential determinants to critically affect olive physiology and health status in a holobiont context (host and its associated organisms). Finally, we have outlined future directions and challenges for xylem microbiome studies based on the recent advances in molecular biology, focusing on metagenomics and culturomics, and bioinformatics network analysis. A better understanding of the xylem olive microbiome will contribute to facilitate the exploration and selection of specific keystone microorganisms that can live in close association with olives under a range of environmental/agronomic conditions. These microorganisms could be ideal targets for the design of microbial consortia that can be applied by endotherapy treatments to prevent or control diseases caused by vascular pathogens or modify the physiology and growth of olive trees.