Using visual cues to perceptually extract sonified data in collaborative, immersive big-data display systems

Abstract

Recently, multi-modal presentation systems have gained much interest to study big data with interactive user groups. One of the problems of these systems is to provide a venue for both personalized and shared information. Especially, sound fields containing parallel audio streams can distract users from extracting necessary information. The way spatial information is processed in the brain allows humans to take complicated visuals and focus on details or the whole. However, temporal information, which can be better presented through audio, is processed differently, making dense sound environments difficult to segregate. In Rensselaer’s CRAIVE-Lab, sounds are presented spatially using an array of 134 loudspeakers to address individual participants who are working on analyzing data together. In this talk, we will present and discuss different methods to improve the ability of participants to focus on their designated audio streams using co-modulated visual cues. In this scheme, the virtual reality space is combined with see-through, augmented reality glasses to optimize the boundaries between personalized and global information. [Work supported by NSF #1229391 and the Cognitive and Immersive Systems Laboratory (CISL).]

From immersion to collaborative embodiment: extending collective interactivity in built environments through human-scale audiovisual immersive technologies

“…Visual information extraction in the immersant space has been used to facilitate perceptual studies on other senses, such as auditory cue retrieval (Lee, Chabot, and Braasch 2017) and the effects of human-scale presentation of architectural renderings on design judgments (Elder 2017). Results in the former experiment discuss the importance of auditory localization on speed and accuracy of extraction, while results in the latter indicate discrepancies in judgments between traditional architectural and humanscale presentations of designs (Chabot et al 2018).…”

Immersive college English teaching design based on big data

“…W Lee, S Chabot and J Braasch put forward that researches in the field of big data mainly focus on visual representation and information extraction, and pay little attention to sound. The aim of this study was to evaluate the most efficient method of extracting visual data using auditory stimulus perception in an immersive environment [1]. Muslem A, Abbas M explored the effects of peer supported immersive multimedia learning strategies on reading and oral production skills.…”