Perceptual history plays an important role in sensory processing and decision making, shaping how we perceive and judge external objects and events. Indeed, past stimuli can bias what we are currently seeing in an attractive fashion, making a current stimulus to appear more similar to its preceding one than it actually is. Such attractive effects across successive stimuli appear to be ubiquitous, affecting almost every aspect of perception, from very basic visual attributes (i.e., orientation) to more complex features (i.e., face identity), suggesting that they may reflect a fundamental principle of brain processing. However, it is unclear whether the ubiquitous nature of these effects is due to an underlying centralised mechanism mediating all of them, or by the existence of separate mechanisms implemented independently in different perceptual pathways. Here we address this question by assessing the behavioural and neural signature of perceptual history in audition and vision, in the context of time perception. Our results first show a double dissociation between the two modalities, whereby the behavioural effect of perceptual history shows opposite patterns of selectivity for the features and position of the stimuli. Electroencephalography results further support a difference between audition and vision, demonstrating that the signature of perceptual history unfolds according to different dynamics in the two modalities and show different relations with the behavioural effect. Overall, our results suggest that the effect of perceptual history may be mediated by different and at least partially independent mechanisms based on the same computational principle, implemented in different sensory pathways.