When we perform an action, its sensory outcomes usually follow shortly after. This characteristic temporal relationship aids in distinguishing self‐ from externally generated sensory input. To preserve this ability under dynamically changing environmental conditions, our expectation of the timing between action and outcome must be able to recalibrate, for example, when the outcome is consistently delayed. Until now, it remains unclear whether this process, known as sensorimotor temporal recalibration, can be specifically attributed to recalibration of sensorimotor (action‐outcome) predictions, or whether it may be partly due to the recalibration of expectations about the intersensory (e.g., audio‐tactile) timing. Therefore, we investigated the behavioral and neural correlates of temporal recalibration and differences in sensorimotor and intersensory contexts. During fMRI, subjects were exposed to delayed or undelayed tones elicited by actively or passively generated button presses. While recalibration of the expected intersensory timing (i.e., between the tactile sensation during the button movement and the tones) can be expected to occur during both active and passive movements, recalibration of sensorimotor predictions should be limited to active movement conditions. Effects of this procedure on auditory temporal perception and the modality‐transfer to visual perception were tested in a delay detection task. Across both contexts, we found recalibration to be associated with activations in hippocampus and cerebellum. Context‐dependent differences emerged in terms of stronger behavioral recalibration effects in sensorimotor conditions and were captured by differential activation pattern in frontal cortices, cerebellum, and sensory processing regions. These findings highlight the role of the hippocampus in encoding and retrieving newly acquired temporal stimulus associations during temporal recalibration. Furthermore, recalibration‐related activations in the cerebellum may reflect the retention of multiple representations of temporal stimulus associations across both contexts. Finally, we showed that sensorimotor predictions modulate recalibration‐related processes in frontal, cerebellar, and sensory regions, which potentially account for the perceptual advantage of sensorimotor versus intersensory temporal recalibration.