Immunomodulatory imide drugs (IMiDs) such as thalidomide, pomalidomide, and lenalidomide represent the most typical cereblon (CRBN) recruiters that are frequently utilized in proteolysis-targeting chimera (PROTAC) design. These CRBN binders, however, cause degradation of IMiD neosubstrates and are innately unstable as they undergo hydrolytic degradation under mild conditions. Here we present the systematic approach towards novel non-phthalimide CRBN binders that were obtained via the simultaneous optimization of their physiochemical properties, stability, on-target affinity, and off-target neosubstrate modulation features. Our efforts led to the discovery of conformationally-locked benzamide-type derivatives that mimic the interactions of the natural CRBN degron, displayed improved chemical stability, and showed a favorable selectivity profile with respect to the recruitment of neosubstrates. The usefulness of the most potent ligands was demonstrated by their conversion into potent degraders of BRD4 and HDAC6 that displayed superiority compared to previously described benchmark PROTACs. We show that our diversified CRBN ligands offer opportunities to design chemically inert proximity-inducing compounds with reduced neomorphic E3 ligase activity of CRBN.