Cognitive function in healthy aging and neurodegenerative diseases like Alzheimer’s disease (AD) correlates to olfactory performance. Aging and disease progression both show marked olfactory deficits in humans and rodents. As a clear understanding of what causes olfactory deficits is still missing, research on this topic is paramount to diagnostics and early intervention therapy. A recent development of this research is focusing on GABAergic interneurons. Both aging and AD show a change in excitation/inhibition balance, indicating reduced inhibitory network functions. In the olfactory system, inhibition has an especially prominent role in processing information, as the olfactory bulb (OB), the first relay station of olfactory information in the brain, contains an unusually high number of inhibitory interneurons. This review summarizes the current knowledge on inhibitory interneurons at the level of the OB and the primary olfactory cortices to gain an overview of how these neurons might influence olfactory behavior. We also compare changes in interneuron composition in different olfactory brain areas between healthy aging and AD as the most common neurodegenerative disease. We find that pathophysiological changes in olfactory areas mirror findings from hippocampal and cortical regions that describe a marked cell loss for GABAergic interneurons in AD but not aging. Rather than differences in brain areas, differences in vulnerability were shown for different interneuron populations through all olfactory regions, with somatostatin-positive cells most strongly affected.