Characterization of Visual Scanning Patterns in Air Traffic Control

Abstract

Characterization of air traffic controllers' (ATCs') visual scanning strategies is a challenging issue due to the dynamic movement of multiple aircraft and increasing complexity of scanpaths (order of eye fixations and saccades) over time. Additionally, terminologies and methods are lacking to accurately characterize the eye tracking data into simplified visual scanning strategies linguistically expressed by ATCs. As an intermediate step to automate the characterization classification process, we (1) defined and developed new concepts to systematically filter complex visual scanpaths into simpler and more manageable forms and (2) developed procedures to map visual scanpaths with linguistic inputs to reduce the human judgement bias during interrater agreement. The developed concepts and procedures were applied to investigating the visual scanpaths of expert ATCs using scenarios with different aircraft congestion levels. Furthermore, oculomotor trends were analyzed to identify the influence of aircraft congestion on scan time and number of comparisons among aircraft. The findings show that (1) the scanpaths filtered at the highest intensity led to more consistent mapping with the ATCs' linguistic inputs, (2) the pattern classification occurrences differed between scenarios, and (3) increasing aircraft congestion caused increased scan times and aircraft pairwise comparisons. The results provide a foundation for better characterizing complex scanpaths in a dynamic task and automating the analysis process.

Designs and Algorithms to Map Eye Tracking Data with Dynamic Multielement Moving Objects

“…The oculomotor statistics on different types of scenarios show that the overall eye fixation numbers and durations on the display (without considering AOIs) did not significantly differ among the scenarios. The results differ from previous aircraft conflict detection research [ 22 , 23 ]. In [ 22 ], eye fixation numbers and durations increased as the difficulty level increased (easy: many aircraft had different altitudes; moderate: many aircraft had similar altitudes; difficult: many aircraft changed altitudes), while setting the number of aircraft on the display at twelve for all scenarios.…”

“…In [ 22 ], eye fixation numbers and durations increased as the difficulty level increased (easy: many aircraft had different altitudes; moderate: many aircraft had similar altitudes; difficult: many aircraft changed altitudes), while setting the number of aircraft on the display at twelve for all scenarios. In [ 23 ], eye fixation numbers and durations increased as the number of aircraft on the display was increased from twelve to twenty. A major difference in the scenario settings was that there was no time limit on detecting possible collisions for [ 22 , 23 ], whereas the experiment in this research had a time limit of twenty minutes.…”

“…In [ 23 ], eye fixation numbers and durations increased as the number of aircraft on the display was increased from twelve to twenty. A major difference in the scenario settings was that there was no time limit on detecting possible collisions for [ 22 , 23 ], whereas the experiment in this research had a time limit of twenty minutes.…”

Scanpath modeling and classification with hidden Markov models

“…For a thorough review of task-prediction algorithms, see (Boisvert and Bruce 2016 ). Eye movements can also be used to quantify mental workload , especially during demanding tasks such as air traffic control (Ahlstrom and Friedman-Berg 2006 ; Di Nocera et al 2006 ; Kang and Landry 2015 ; McClung and Kang 2016 ; Mannaru et al 2016 ). Another very promising line of studies is gaze-based disease screening (Itti 2015 ).…”

An eye-hand movement Model for traffic complexity evaluation and monitoring enhancement for air traffic controllers

“…Analysis of how ATCOs looked at the aircraft (an object on the display) was performed after the simulation, by studying which and how aircraft are scanned together with their pairing and grouping from the ATCO's eye ball movement [59,60]. Further algorithms were then developed to compute and classify the scanpath strategies of ATCOs, in order to characterise their visual scanning patterns [61,62]. Their individual performance determined using the duration of scanning required before conflict detection was derived.…”