Visual Analytics Tools for Polycode Stimuli Eye Gaze Tracking in Virtual Reality

Abstract

This paper is devoted to extending the previously created unified pipeline for conducting eye-tracking- based experiments in a virtual reality environment. In the previous work, we proposed using SciVi semantic data mining platform, Unreal Engine and HTC Vive Pro Eye head-mounted display to study reading process in the immersive virtual reality. The currently proposed extension enables to handle so-called polycode stimuli: compound visual objects, which consist of individual parts carrying different semantics for the viewer. To segment polycode stimuli extracting areas of interest (areas, where the informant’s eye gaze is being tracked) we adopt Creative Maps Studio vector graphics editor. To integrate Creative Maps Studio into the existing pipeline we created plugins for SciVi platform to load and handle the segmented stimuli, place them in the virtual reality scenes, collect corresponding eye gaze tracking data and perform visual analysis of the data collected. To analyze the eye gaze tracks, we utilize a circular graph that allows comprehensive visualization of hierarchical areas of interest (mapping them to color- coded graph nodes grouped into the hierarchy with a help of multilevel circular scale) and corresponding eye movements (mapped to the graph edges). We tested our pipeline on two different stimuli: the advertising poster and the painting “The Appearance of Christ Before the People” by A. Ivanov (1857).

Eye Tracking Data Mining Based on Fuzzy Sets of Fixations

“…The informants' eye gaze tracks were recorded by an embedded Tobii eye tracker. Each poster was manually segmented into AOIs in the Creative Maps Studio vector editor [9]. Then, the experimental data were analyzed in the SciVi platform using the data mining algorithm shown in Fig.…”

“…For this, an algorithm suggested by J. Llanes-Jurado et al is used [22]. The "Segmented Map" operator loads the descriptions of AOIs for the visual stimulus in question (the segmentation into AOIs is done in the Creative Maps Studio vector graphics editor as described in [9]). The "Scanpath Merger" operator assembles the fuzzy scanpath according to equations ( 1) and ( 2).…”

“…These challenges are indicated as belonging to the major ones of state-of-the-art eye tracking research methodology [2]. We implemented the proposed approach within our visual analytics platform SciVi (https://scivi.tools/) [8] and tested it in practical use cases within a remit of a research project devoted to studying the peculiarities of multimodal information perception in a virtual reality (VR) environment [9].…”