Mediterranean landscapes were drastically affected by high levels of abandonment of agricultural and other land practices during the last century. These changes in land use can have significant effects on diversity patterns by altering disturbance and competition equilibria within plant communities at the landscape level. Particularly, such changes have been found to affect the patterns of phylogenetic diversity and structure by causing nonrandom losses of species through filtering effects and landscape homogenization. By investigating diversity patterns across a region submitted to high levels of land use changes, located in a (sub‐) mountainous area of northwestern Greece (northern Pindus), we aimed at understanding the patterns of phylogenetic diversity and structure in relation to land abandonment and the subsequent recovery of natural vegetation. We sampled 250 vegetation plots equally divided in grasslands and forests, distributed across the different classes of land use occurring in the general study area based on the period since the last change in land use. Standardized metrics of Faith's phylogenetic diversity, mean phylogenetic distance, and mean nearest taxon distance were used to investigate phylogenetic diversity patterns across communities and different land‐use regimes. A Principal Coordinates of Phylogenetic Structure analysis was employed to evaluate the variation in lineage composition among communities, and boosted regression trees were used to identify the relative influence of community differentiation (as captured by the classification of sampling plots in ecologically and floristically distinct vegetation communities), plant life strategies (competition, stress tolerance, and disturbance), and climatic, topographic, and soil variables on phylogenetic diversity metrics. Community differentiation was identified as the main driver of phylogenetic patterns. Additionally, phylogenetic diversity and structure were observed as having a statistically significantly negative correlation with disturbance, a statistically significantly positive correlation with stress tolerance, and a weaker positive correlation with competition. Phylogenetic clustering was observed for the early successional grassland communities submitted to stronger effects of disturbance, while phylogenetic randomness (or rarely overdispersion) was observed in forest communities submitted to stronger effects of competition. Finally, phylogenetic clustering of grassland communities was more evident shortly after land abandonment.