Neuronal loss is the best neuropathological substrate that correlates with cortical atrophy and dementia in Alzheimer's disease (AD). Defective GABAergic neuronal functions may lead to cortical network hyperactivity and aberrant neuronal oscillations and in consequence, generate a detrimental alteration in memory processes. In this study, using immunohistochemical and stereological approaches, we report that the two major and non‐overlapping groups of inhibitory interneurons (SOM‐cells and PV‐cells) displayed distinct vulnerability in the perirhinal cortex of APP/PS1 mice and AD patients. SOM‐positive neurons were notably sensitive and exhibited a dramatic decrease in the perirhinal cortex of 6‐month‐old transgenic mice (57% and 61% in areas 36 and 35, respectively) and, most importantly, in AD patients (91% in Braak V–VI cases). In addition, this interneuron degenerative process seems to occur in parallel, and closely related, with the progression of the amyloid pathology. However, the population expressing PV was unaffected in APP/PS1 mice while in AD brains suffered a pronounced and significant loss (69%). As a key component of cortico‐hippocampal networks, the perirhinal cortex plays an important role in memory processes, especially in familiarity‐based memory recognition. Therefore, disrupted functional connectivity of this cortical region, as a result of the early SOM and PV neurodegeneration, might contribute to the altered brain rhythms and cognitive failures observed in the initial clinical phase of AD patients. Finally, these findings highlight the failure of amyloidogenic AD models to fully recapitulate the selective neuronal degeneration occurring in humans.