Semi‐natural grasslands are found predominantly in agricultural landscapes; however, their counterparts are also found in cities, referred to as urban grasslands. We sampled the most extensive patches of urban grasslands in Wrocław (Poland, Europe) in urban parks and along a river valley. We compared urban grasslands to broadly defined rural grasslands in terms of compositional gradients and both taxonomic and functional diversity. We also tested for relationships between rare species and different facets of diversity. Urban grasslands corresponded to three rural grassland habitat types: mesic pastures, mesic meadows, and alluvial meadows. We conducted analogous analyses, including and excluding habitat‐level assignment, to determine if emerging patterns between rural and urban grasslands are habitat‐dependent. Rural pastures and their urban counterparts showed high taxonomic and functional similarity, implying that short‐ and frequent‐cutting, applied in cities, mimics grazing management on agricultural sites. Rural mesic hay grasslands and alluvial meadows showed high taxonomic and functional dissimilarities from their urban counterparts. We found that rare species contribute to taxonomic diversity while only affecting functional diversity in a few cases. Varying patterns of taxonomic and functional similarity were observed across different habitats. Comparisons of grasslands without regard for habitat types mask diverging patterns in taxonomic and functional diversity. Despite high environmental and management pressures in urban areas (higher nutrient levels and temperatures, decreased moisture, and intensified management and soil disturbances), we showed that urban grasslands hold a considerable portion of the plant diversity found in rural grasslands.