Agricultural intensification poses a major threat to the conservation of biodiversity and associated ecosystem services. Since non-crop habitats are regarded as important refuges for farmland biodiversity, various greening measures have been proposed to halt biodiversity loss. However, the effectiveness of these measures for biodiversity conservation is still under debate. Therefore, we here compared ground-dwelling beetle (Coleoptera) assemblages of different non-crop habitats (field margins, set-aside fields sown with wildflowers, and permanent grassland fallows) and wheat fields within an intensively used agricultural landscape in western Germany. Taxonomic diversity of Carabidae, Staphylinidae and other coleopteran families and their conservation value were higher in all non-crop habitats than on wheat fields. Surprisingly, though, different types of non-crop habitats did not differ in species richness or the number of threatened species. Thus, field margins and sown wildflower fields were as effective in promoting beetle diversity as grassland fallows. However, different non-crop habitats supported different species assemblages, and several species, in particular especially large ones, were restricted to grassland fallows. These results suggest that different greening measures are effective in promoting the biodiversity of beetles, and that permanent grassland fallows are essential for nature conservation. The fact that habitat types harbored different assemblages stresses the need to combine a variety of greening measures to yield the highest benefit for biodiversity.