Visual adaptation illusions indicate that our perception is influenced not only by the current stimulus but also by what we have seen in the recent past. Adaptation to stimulus contrast (the relative luminance created by edges or contours in a scene) induces the perception of the stimulus fading away and increases the contrast detection threshold in psychophysical tests [1, 2]. Neural correlates of contrast adaptation have been described throughout the visual system including the retina [3], dorsal lateral geniculate nucleus (dLGN) [4, 5], primary visual cortex (V1) [6], and parietal cortex [7]. The apparent ubiquity of adaptation at all stages raises the question of how this process cascades across brain regions [8]. Focusing on V1, adaptation could be inherited from pre-cortical stages, arise from synaptic depression at the thalamo-cortical synapse [9], or develop locally, but what is the weighting of these contributions? Because contrast adaptation in mouse V1 is similar to classical animal models [10, 11], we took advantage of the optogenetic tools available in mice to disentangle the processes contributing to adaptation in V1. We disrupted cortical adaptation by optogenetically silencing V1 and found that adaptation measured in V1 now resembled that observed in dLGN. Thus, the majority of adaptation seen in V1 neurons arises through local activity-dependent processes, with smaller contributions from dLGN inheritance and synaptic depression at the thalamo-cortical synapse. Furthermore, modeling indicates that divisive scaling of the weakly adapted dLGN input can predict some of the emerging features of V1 adaptation.