The natural forest in southwest Ethiopia is progressively modified to coffee agroforest. To this effect forest composition and diversity is simplified to local preferred coffee shade trees. Woody plant species that are less managed require the conservation priority in coffee agroforest. The study aims at assessing diversity of plant species, investigating local people preference and finally identify woody plants for conservation priority in coffee agroforest in southwest Ethiopia. Data were collected on ecological and ethnoecological information through field assessment and individual interview. Vegetation data were collected from 63 plots distributed across five sites. Ethnoecological data were collected from 96 individuals across five villages living adjacent to the forest through semi-structured interview. The result showed that 48 different woody plant species belonging to 27 families were recorded. Most of the families were represented by single species. The regeneration status of these woody plant species are unsatisfactory or poor. Three species; Cordia africana, Croton macrostachyus, and Milletia ferruginea were accounting for 41 percent of the total number of woody plant species in coffee agroforest. The aggregate relative preference score showed 15 most preferred woody plant species in coffee agroforest. The use value of these species were cited mainly for timber, hanging beehive and beehive making than coffee shade. The findings suggest that 12 woody plants need high conservation priority, 19 species need moderate conservation priority and 17 woody plants need low priority for conservation. The Spearman correlation showed negative correlation between woody plant abundance and conservation priority [rs (46) = −0.681, p = 000]. The study findings suggest that woody plant conservation priority in coffee agroforest should take into consideration local preference of woody plant species.