Interactive Fixation-to-AOI Mapping for Mobile Eye Tracking Data based on Few-Shot Image Classification

Abstract

Mobile eye tracking is an important tool in psychology and humancentred interaction design for understanding how people process visual scenes and user interfaces. However, analysing recordings from mobile eye trackers, which typically include an egocentric video of the scene and a gaze signal, is a time-consuming and largely manual process. To address this challenge, we propose a web-based annotation tool that leverages few-shot image classification and interactive machine learning (IML) to accelerate the annotation process. The tool allows users to efficiently map fixations to areas of interest (AOI) in a video-editing-style interface. It includes an IML component that generates suggestions and learns from user feedback using a few-shot image classification model initialised with a small number of images per AOI. Our goal is to improve the efficiency and accuracy of fixation-to-AOI mapping in mobile eye tracking.

Quantifying dwell time with location-based augmented reality: Dynamic AOI analysis on mobile eye tracking data with vision transformer

“…Silva Machado et al (2019) matched the detected bounding boxes with participants' fixations using a sliding-window approach with a MobileNet CNN model. Rather than using the model for inference on large data for automatic analysis, several tools are multitask and offer interactive visualization for manual annotation (Barz et al, 2023;Kurzhals, 2021;Kurzhals et al, 2017Kurzhals et al, , 2020Panetta et al, 2019). Kurzhals et al (2017) an interactive labeling tool with automatic clustering combined with an analysis system.…”

“…They went on to develop image-based (Kurzhals et al, 2020) and gaze patches techniques (Kurzhals, 2021) for dynamic AOI annotation that are conceptually similar to our proposed idea of merging gaze data with video. Barz et al (2023) have implemented an approach based both on image classification and object detection. They used a few-shot learning method for its adaptability, with a 50-layer CNN (ResNet50).…”

“…Barz et al. ( 4 ) have implemented an approach based both on image classification and object detection. They used a few-shot learning method for its adaptability, with a 50-layer CNN (ResNet50).…”