Embodied interaction with complex neuronal data in mixed-reality

Betella, Alberto; Carvalho, Rodrigo; Sanchez-Palencia, Jesus; Bernardet, Ulysses; Verschure, Paul F. M. J.

doi:10.1145/2331714.2331718

Cited by 13 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a proof of principle, the XIM has been previously been tested for visualization and analysis of artificially generated networks (Betella et al 2012). The system transforms neuronal systems designed with iqr into three-dimensional representations of neuronal processes, groups and connections.…”

Section: Methodsmentioning

confidence: 99%

The Dynamic Connectome: A Tool For Large-Scale 3D Reconstruction Of Brain Activity In Real-Time

Arsiwalla¹,

Betella²,

Bueno³

et al. 2013

ECMS 2013 Proceedings Edited By: Webjorn Rekdalsbakken, Robin T. Bye, Houxiang Zhang

Self Cite

View full text Add to dashboard Cite

We present a large-scale simulation tool for real-time 3D reconstruction of brain activity in a virtual reality environment. The 3D interactive visualization of the human cortex connectome in virtual reality is achieved by using a gaming engine (Unity 3D). Further, the visualization is bi-directionally interfaced with a real-time neuronal simulator, iqr. As a result, by stimulating populations of neurons with external input currents, we are able to reconstruct neural activity propagating in 3D and in real-time. We explicitly demonstrate causal activity propagation in the parietal lobe, indicative of visuo-motor integration. This is a first step to simulating and mapping large-scale brain activity.

show abstract

Section: Methodsmentioning

confidence: 99%

The Dynamic Connectome: A Tool For Large-Scale 3D Reconstruction Of Brain Activity In Real-Time

Arsiwalla¹,

Betella²,

Bueno³

et al. 2013

ECMS 2013 Proceedings Edited By: Webjorn Rekdalsbakken, Robin T. Bye, Houxiang Zhang

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is derived from the large-scale multimodal interactive exhibition, Ada, visited by over 500,000 people (Eng et al, 2002) and has been used to study human navigation (Betella, Bueno, Bernardet, & Verschure, 2013), cooperation, and social salience (Inderbitzin, Betella, Lanatá, Scilingo, Bernardet, & Verschure, 2013), and the exploration of complex data sets (Betella, Carvalho, Sanchez-Palencia, Bernardet, & Verschure, 2012).…”

Section: The Experience Induction Machinementioning

confidence: 99%

Subliminal Response Priming in Mixed Reality: The Ecological Validity of a Classic Paradigm of Perception

Cetnarski

Betella

Prins

et al. 2014

Presence: Teleoperators and Virtual Environments

Self Cite

View full text Add to dashboard Cite

Subliminal stimuli can affect perception, decision-making, and action without being accessible to conscious awareness. Most evidence supporting this notion has been obtained in highly controlled laboratory conditions. Hence, its generalization to more realistic and ecologically valid contexts is unclear. Here, we investigate the impact of subliminal cues in an immersive navigation task using the so-called eXperience Induction Machine (XIM), a human accessible mixed-reality system. Subjects were asked to navigate through a maze at high speed. At irregular intervals, one group of subjects was exposed to subliminal aversive stimuli using the masking paradigm. We hypothesized that these stimuli would bias decision-making. Indeed, our results confirm this hypothesis and indicate that a subliminal channel of interaction exists between the user and the XIM. These results are relevant in our understanding of the bandwidth of communication that can be established between humans and their physical and social environment, thus opening up to new and powerful methods to interface humans and artefacts.

show abstract

“…On a first stage a Neuroscience application was designed interfacing the XIM visualization with large-scale computational neuronal networks developed with the Iqr simulation software [4]. With this version we were able to show in realtime the neuronal activity of the running simulation and allowed the active exploration of the intricate organization and connectivity of the networks through real-time interaction in the XIM [5]. We are extending the Neuroscience application to represent the human brain structural connectome [13], a large dataset that stems from the analysis of the connectivity of the human neocortex.…”

Section: Applicationmentioning

confidence: 99%

A sensing architecture for empathetic data systems

Wagner

Lingenfelser

André

et al. 2013

Proceedings of the 4th Augmented Human International Conference

Self Cite

View full text Add to dashboard Cite

Today's increasingly large and complex databases require novel and machine aided ways of exploring data. To optimize the selection and presentation of data, we suggest an unconventional approach. Instead of exclusively relying on explicit user input to specify relevant information or to navigate through a data space, we exploit the power and potential of the users' unconscious processes in addition. To this end, the user is immersed in a mixed reality environment while his bodily reactions are captured using unobtrusive wearable devices. The users' reactions are analyzed in real-time and mapped onto higher-level psychological states, such as surprise or boredom, in order to trigger appropriate system responses that direct the users' attention to areas of potential interest in the visualizations. The realization of such a close experience-based human-machine loop raises a number of technical challenges, such as the real-time interpretation of psychological user states. The paper at hand describes a sensing architecture for empathetic data systems that has been developed as part of such a loop and how it tackles the diverse challenges.

show abstract

Embodied interaction with complex neuronal data in mixed-reality

Cited by 13 publications

References 16 publications

The Dynamic Connectome: A Tool For Large-Scale 3D Reconstruction Of Brain Activity In Real-Time

The Dynamic Connectome: A Tool For Large-Scale 3D Reconstruction Of Brain Activity In Real-Time

Subliminal Response Priming in Mixed Reality: The Ecological Validity of a Classic Paradigm of Perception

A sensing architecture for empathetic data systems

Contact Info

Product

Resources

About