USP5 is a deubiquitinase that has been implicated in a range of diseases, including cancer, but no USP5-targeting chemical probe has been reported to date. Here, we present the progression of a chemical series that occupies the C-terminal ubiquitin-binding site of a poorly characterized zinc-finger ubiquitin binding domain (ZnF-UBD) of USP5 and allosterically inhibits the catalytic activity of the enzyme. Systematic exploration of the structure-activity relationship, complemented with crystallographic characterization of the ZnF-UBD bound to multiple ligands, led to the identification of 64, which binds to the USP5 ZnF-UBD with a KD of 2.8 μM. 64 is selective over the structurally similar ZnF-UBD domain of HDAC6 and inhibits USP5 catalytic activity in vitro with an IC50 of 26 μM. This study provides a chemical and structural framework for the discovery of a chemical probe to delineate USP5 function in cells.