The recently discovered SAFit class of inhibitors against the Hsp90 co‐chaperone FKBP51 show greater than 10 000‐fold selectivity over its closely related paralogue FKBP52. However, the mechanism underlying this selectivity remained unknown. By combining NMR spectroscopy, biophysical and computational methods with mutational analysis, we show that the SAFit molecules bind to a transient pocket in FKBP51. This represents a weakly populated conformation resembling the inhibitor‐bound state of FKBP51, suggesting conformational selection rather than induced fit as the major binding mechanism. The inhibitor‐bound conformation of FKBP51 is stabilized by an allosteric network of residues located away from the inhibitor‐binding site. These residues stabilize the Phe67 side chain in a dynamic outward conformation and are distinct in FKBP52, thus rationalizing the basis for the selectivity of SAFit inhibitors. Our results represent a paradigm for the selective inhibition of transient binding pockets.