Inattention-Management Middleware for Human-in-the-Loop Multi-Display Applications

Abstract

Operator inattention is an important and unsolved problem in mission critical multi-display systems where a single or a group of operators continuously monitor information flows on distributed displays. In this paper we present a novel system solution to this problem and a middleware for supporting flexible attention-aware applications for a variety of domains. Some of the most significant functionality includes direct querying of the application's attention state, custom callback definitions to be executed on specific attention events or application updates, inter-application message routing, and pushing custom notification with relative location information to any other registered application. We evaluate our middleware by developing three applications that both demonstrate the efficacy and versatility of the system and provide performance estimates in terms of latency as a function of payload size. Index Terms-inattention management, human-in-the-loop, multi-display, middleware

A Conceptual Design of an Inattention Management Middleware with Adaptive Target Saliency

“…Garrido et al [17] later distilled these research ideas into a toolkit with attention-aware graphical user interface (GUI) controls. [18] developed the Inattention Management Middleware (IMM), which provides several of the features previously discussed, such as fusing sensor data, detecting operator's focus of attention, detecting operators' target fixation and providing an interface for an application to specify callbacks on specific user events. The IMM provides an API that allows web-applications to detect whether specific data points have been looked at and allows the definition of application callbacks to be triggered on detection and data point intersection events.…”

“…Their positions, heights and tilts were derived experimentally as to minimize interference between the eye-trackers. The IMM used in this formative was a more advanced version of our first prototype [18].…”