Virtual reality allows for testing of multisensory integration for balance using portable Head Mounted Displays (HMDs). HMDs provide head kinematics data while showing a moving scene when participants are not. Are HMDs useful to investigate postural control? We used an HMD to investigate postural sway and head kinematics changes in response to auditory and visual perturbations and whether this response varies by context. We tested 25 healthy adults, and a small sample of people with diverse monaural hearing (n = 7), or unilateral vestibular dysfunction (n = 7). Participants stood naturally on a stable force-plate and looked at 2 environments via the Oculus Rift (abstract “stars;” busy “street”) with 3 visual and auditory levels (static, “low,” “high”). We quantified medio-lateral (ML) and anterior-posterior (AP) postural sway path from the center-of-pressure data and ML, AP, pitch, yaw and roll head path from the headset. We found no difference between the different combinations of “low” and “high” visuals and sounds. We then combined all perturbations data into “dynamic” and compared it to the static level. The increase in path between “static” and “dynamic” was significantly larger in the city environment for: Postural sway ML, Head ML, AP, pitch and roll. The majority of the vestibular group moved more than controls, particularly around the head, when the scenes, especially the city, were dynamic. Several patients with monaural hearing performed similar to controls whereas others, particularly older participants, performed worse. In conclusion, responses to sensory perturbations are magnified around the head. Significant differences in performance between environments support the importance of context in sensory integration. Future studies should further investigate the sensitivity of head kinematics to diagnose vestibular disorders and the implications of aging with hearing loss to postural control. Balance assessment and rehabilitation should be conducted in different environmental contexts.