Inhibition of the human O-linked β-N-acetylglucosaminidase (hOGA, GH84) enzyme is pharmacologically relevant in several diseases such as neurodegenerative and cardiovascular disorders, type 2 diabetes, and cancer. Human lysosomal hexosaminidases (hHexA and hHexB, GH20) are mechanistically related enzymes; therefore, selective inhibition of these enzymes is crucial in terms of potential applications. In order to extend the structure–activity relationships of OGA inhibitors, a series of 2-acetamido-2-deoxy-d-glucono-1,5-lactone sulfonylhydrazones was prepared from d-glucosamine. The synthetic sequence involved condensation of N-acetyl-3,4,6-tri-O-acetyl-d-glucosamine with arenesulfonylhydrazines, followed by MnO2 oxidation to the corresponding glucono-1,5-lactone sulfonylhydrazones. Removal of the O-acetyl protecting groups by NH3/MeOH furnished the test compounds. Evaluation of these compounds by enzyme kinetic methods against hOGA and hHexB revealed potent nanomolar competitive inhibition of both enzymes, with no significant selectivity towards either. The most efficient inhibitor of hOGA was 2-acetamido-2-deoxy-d-glucono-1,5-lactone 1-naphthalenesulfonylhydrazone (5f, Ki = 27 nM). This compound had a Ki of 6.8 nM towards hHexB. To assess the binding mode of these inhibitors to hOGA, computational studies (Prime protein–ligand refinement and QM/MM optimizations) were performed, which suggested the binding preference of the glucono-1,5-lactone sulfonylhydrazones in an s-cis conformation for all test compounds.