The goal of the proposed experiments is to assess the fidelity of a simulation as well as develop synthetic imaging systems which will result in controlled functional Magnetic Resonance Imaging (fMRI) experiments involving the normal population as well as patients. The ultimate goal of the experiments planned is to explore whether natural and artificial scenes of varied fidelity for training or for therapeutic purposes engage common perceptual or neuroscientific mechanisms. Such input is non-obtrusive and is derived at the same time as the experience occurs. The first experiment briefly summarized is exploring brain empathic responses of users participating in an interactive economic game developed to be displayed in a fMRI scanner. The second experiment presents an interactive lighting system also to be displayed in an fMRI scanner. Its scope is to enable the monitoring of neural activation patterns in response to lighting manipulations ranging from daylight and artificial light of the normal population initially and potentially of groups of patients suffering from the depersonalization syndrome. A broader aim of this work is to assess whether such powerful socialpsychological studies could be usefully carried out within Virtual Environments advancing both cognitive neuroscience and computer graphics research.
Keywords-computer graphics simulations; perceptuallybased rendering; fidelity metrics; cognitive neuroscience
I. SCOPEVirtual environment (VE) and Simulation technology is an advancing medium that allows its users to interact with computer-generated three-dimensional worlds. These worlds could be displayed on a typical desktop monitor or onto more sophisticated immersive displays such as Head Mounted Displays (HMDs) or even functional Magnetic Resonance Imaging (fMRI) scanners. What sets VE technology apart from its ancestors is that in VE systems for simulation and training users receive multi-sensory stimuli which respond to their reactions (for instance, using head tracking) and are intended to provide a sensation of natural interaction with the synthetic world, aiming at positive transfer of training in real-world, often hazardous, task situation such training in a flight simulator. While there has been extensive work to enhance the perceptual realism of VEs and graphic displays, it is never clear to what extent such improvements in visual quality are associated with differences in visual cognition of the trainees. For example, recent computer graphics technology allows very high quality graphics to be produced -but how much could the quality be reduced without consequent changes in the cognitive processes in the human observer? This may depend on the importance of the subset of the scene and its scope which drives human perception at that instant. Thus, for example, the eyes of a synthetic character are important in revealing their owner's intentions and are therefore in need of high-quality rendering. One of the purposes of the proposed work is to understand how the brain processes perceptually impor...