Human glutaminyl cyclase (hQC) is drawing considerable attention and emerging as a potential druggable target for Alzheimer's disease (AD) due to its close involvement in the pathology of AD via the post-translational pyroglutamate modification of amyloid-β. A recent phase 2a study has shown promising early evidence of efficacy for AD with a competitive benzimidazole-based QC inhibitor, PQ912, which also demonstrated favorable safety profiles. This finding has sparked new hope for the treatment of AD. In this review, we briefly summarize the discovery and evolution of hQC inhibitors, with a particular interest in classic Zinc binding group (ZBG)-containing chemicals reported in recent years. Additionally, we highlight several high-potency inhibitors and discuss new trends and challenges in the development of QC inhibitors as an alternative and promising disease-modifying therapy for AD.