Design and Application of Real-Time Visual Attention Model for the Exploration of 3D Virtual Environments

Abstract

Abstract-This paper studies the design and application of a novel visual attention model meant to compute users gaze position automatically, i.e. without using a gaze-tracking system. The model we propose is specifically designed for real-time first-person exploration of 3D virtual environments. It is the first model adapted to this context which can compute, in real-time, a continuous gaze point position instead of a set of 3D objects potentially observed by the user. To do so, contrary to previous models which use a mesh-based representation of visual objects, we introduce a representation based on surface-elements. Our model also simulates visual reflexes and the cognitive process which takes place in the brain such as the gaze behavior associated to first-person navigation in the virtual environment. Our visual attention model combines the bottom-up and top-down components to compute a continuous gaze point position on screen that hopefully matches the user's one. We conducted an experiment to study and compare the performance of our method with a state-of-the-art approach. Our results are found significantly better with more than 100% of accuracy gained. This suggests that computing in real-time a gaze point in a 3D virtual environment is possible and is a valid approach as compared to object-based approaches. Finally, we expose different applications of our model when exploring virtual environments. We present different algorithms which can improve or adapt the visual feedback of virtual environments based on gaze information. We first propose a level-of-detail approach that heavily relies on multipletexture sampling. We show that it is possible to use the gaze information of our visual attention model to increase visual quality where the user is looking, while maintaining a high refresh rate. Second, we introduce the use of visual attention model in three visual effects inspired from the human visual system namely: depth-of-field blur, camera motions, and dynamic luminance. All these effects are computed based on simulated user's gaze, and are meant to improve user's sensations in future virtual reality applications.Index Terms-visual attention model, first person exploration, gaze tracking, visual effects, level of detail.

Inaccuracy-Tolerant Sparse-to-Dense Depth Propagation for Semiautomatic 2D-to-3D Conversion

“…where I i denotes the Lab color values at pixel i. e reason for using Lab color space is that it takes into account human perception [22]. N(i) is the set of 8 neighbors of pixel i. δ(·) is the Dirac delta function.…”

Evaluation of Attention‐Guiding Video Visualization

“…By calculating saliency maps based on bottom‐up features such as intensity, color, orientation, or motion, attention can be guided by semantic depth of field [ST10] or subtle image modulation [VMFS11]. Hillaire et al [HLRC*12] provided an attention model for 3D environments that combines bottom‐up and top‐down features in order to predict human gaze behavior. Bonanni et al [BLS05] created user‐centered interfaces for an augmented reality kitchen with consideration of the user's attentional focus.…”

A influência da percepção visual de movimento sobre desempenho em tarefa tomada de decisão

“…Apesar deste amplo uso da RV, o potencial desta técnica para o estudo da atenção e percepção visual não foi ainda explorado. Alguns relatos iniciais sugerem que a RV pode ser um método confiável para o estudo da atenção (Hillaire et al, 2011;Dvorkin, Bogey, Harvey & Patton, 2011) e efeito de medicamentos, por exemplo (Pollak, Shomaly, Weiss, Rizzo & Gross-Tsur, 2010). Uma grande vantagem que a RV oferece é a possibilidade da criação de inúmeras situações experimentais sem abrir mão do rigor da obtenção dos dados em laboratório, além de permitir o controle da intensidade da exposição do indivíduo (Falkenstein, Ramos, 2013).…”