The feeling of control over one’s actions, termed the Sense of Agency (SoA), delineates one’s experience as an embodied self. Although, this embodied experience is typically perceived as stable over time, recent theoretical accounts highlight the experience-dependent and dynamic nature of the embodied self. In this study we examined how recent experiences modulate SoA (i.e., serial dependence), and disambiguated the unique contributions of previous stimuli and choices on subsequent SoA judgments. In addition, we examined whether these effects persist across different domains of perceptual alteration. We analyzed two independent datasets of the Virtual Hand (VH) task (N = 100 participants) in which a sensorimotor conflict is introduced between the presented visual feedback and the actual movement performed. In Dataset 1, which included only temporal alterations, we found that previous stimuli recalibrate current perception, increasing the likelihood of the current choice to be different than the previous choice. Whereas previous choices induce a repetition bias increasing the likelihood to repeat choices across trials. Thus, previous external stimuli and self-generated choices exert opposing influences on SoA. We replicated these findings in Dataset 2, in which the VH task was tested with alterations in both temporal and spatial domains. In addition, we discovered that previous stimuli from a different perceptual domain exert a recalibration effect similar to stimuli from the same domain. Thus, SoA is constantly shaped by our previous subjective choices and objective stimuli experienced even across different perceptual domains. This highlights how SoA may act as unifying construct organizing our experience of the self over time and across perceptual experiences.