Invasive alien species are considered the second greatest global threat to biodiversity. This study is aimed at determining the impacts of Parthenium hysterophorus on herbaceous and woody plant species diversity in the Ginir district, southeast Ethiopia. Data on vegetation were collected from the three study sites’ four land use types, with each land use type having invaded and non-invaded land units. A systematic random sampling method was used for establishing sampling plots. To examine the impacts of the invasive on native plant diversity, a total of 160 plots (120 plots of 1 m2 on grazing lands, roadsides, and abandoned agricultural lands and 40 plots of 20 m2 for sampling herbaceous and tree (shrub) species, respectively) were established. The number of plots was equally distributed in both invaded and adjacent non-invaded areas. Plant species from each plot were recorded and identified. In each plot, all the individuals of P. hysterophorus were counted, the heights of the five tallest individuals were measured, and the mean height was calculated. The percentage cover of P. hysterophorus was visually estimated. The data were analyzed using both descriptive and inferential statistics. A total of 105 plant species (45 trees/shrubs and 60 herbaceous) belonging to 84 genera and 42 families were documented in the study area. The result showed a strong negative relationship between the density of P. hysterophorus and other plant species richness (r = −0.82, p = 0.013) and species abundance (r = −0.917, p = 0.001) per study site of the invaded community. Species richness in the non-invaded site was higher (105 species) than in the invaded area (63 species), demonstrating the negative impact of P. hysterophorus on local biodiversity. Furthermore, the number of plant families was 42 in the non-invaded area, in contrast to only 32 in the invaded areas, a 23.8% decline. Of the plant communities, similarity indices between non-invaded and invaded sites among different land use types were >50%. It was concluded that P. hysterophorus was one of the most dominant invasive alien species in the study area that reduced the species diversity of various plant species. Putting in place a strategy and effective planning for the control and management of this invasive alien species is strongly recommended.