Filtering out familiar, irrelevant stimuli eases the computational burden on the cerebral cortex. Inhibition is a candidate mechanism in this filtration process. Oscillations in the cortical local field potential (LFP) serve as markers of the engagement of different inhibitory neurons. In awake mice, we find pronounced changes in LFP oscillatory activity present in layer 4 of primary visual cortex (V1) with progressive stimulus familiarity. Over days of repeated stimulus presentation, low frequency (alpha/beta ∼15 Hz peak) power increases while high frequency (gamma ∼65 Hz peak) power decreases. This high frequency activity re-emerges when a novel stimulus is shown. Thus, high frequency power is a marker of novelty while low frequency power signifies familiarity. Two-photon imaging of neuronal activity reveals that parvalbumin-expressing inhibitory neurons disengage with familiar stimuli and reactivate to novelty, consistent with their known role in gamma oscillations, whereas somatostatin-expressing inhibitory neurons show opposing activity patterns, indicating a contribution to the emergence of lower frequency oscillations. We also reveal that stimulus familiarity and novelty have differential effects on oscillations and cell activity over a shorter timescale of seconds. Taken together with previous findings, we propose a model in which two interneuron circuits compete to drive familiarity or novelty encoding.