Every instant of perception depends on a cascade of brain processes calibrated to the history of sensory and decisional events. In the present work, we show that human visual perception is constantly shaped by two contrasting forces exerted by sensory adaptation and past decisions. In a series of experiments, we used multilevel modeling and cross-validation approaches to investigate the impact of previous stimuli and decisions on behavioral reports during adjustment and forced-choice tasks. Our results revealed that each perceptual report is permeated by opposite biases from a hierarchy of serially dependent processes: Low-level adaptation repels perception away from previous stimuli, whereas decisional traces attract perceptual reports toward the recent past. In this hierarchy of serial dependence, “continuity fields” arise from the inertia of decisional templates and not from low-level sensory processes. This finding is consistent with a Two-process model of serial dependence in which the persistence of readout weights in a decision unit compensates for sensory adaptation, leading to attractive biases in sequential perception. We propose a unified account of serial dependence in which functionally distinct mechanisms, operating at different stages, promote the differentiation and integration of visual information over time.