Virtual Reality is expanding its use to several fields of application, including health and education. The continuous growth of this technology comes with new challenges related to the ways in which users feel inside these virtual environments. There are various guidelines on ways to enhance users’ virtual experience in terms of immersion or presence. Nonetheless, there is no extensive research on enhancing the sense of agency (SoA), a phenomenon which refers to the self-awareness of initiating, executing, and controlling one’s actions in the world. After reviewing the state of the art of technologies developed in the field of Affective Computing (AC), we propose a framework for designing immersive virtual environments (IVE) to enhance the users’ SoA. The framework defines the flow of interaction between users and the virtual world, as well as the AC technologies required for each interactive component to recognise, interpret and respond coherently within the IVE in order to enhance the SoA.