Brain–Computer Interface-Based Adaptive Automation to Prevent Out-Of-The-Loop Phenomenon in Air Traffic Controllers Dealing With Highly Automated Systems

Abstract

Increasing the level of automation in air traffic management is seen as a measure to increase the performance of the service to satisfy the predicted future demand. This is expected to result in new roles for the human operator: he will mainly monitor highly automated systems and seldom intervene. Therefore, air traffic controllers (ATCos) would often work in a supervisory or control mode rather than in a direct operating mode. However, it has been demonstrated how human operators in such a role are affected by human performance issues, known as Out-Of-The-Loop (OOTL) phenomenon, consisting in lack of attention, loss of situational awareness and de-skilling. A countermeasure to this phenomenon has been identified in the adaptive automation (AA), i.e., a system able to allocate the operative tasks to the machine or to the operator depending on their needs. In this context, psychophysiological measures have been highlighted as powerful tool to provide a reliable, unobtrusive and real-time assessment of the ATCo’s mental state to be used as control logic for AA-based systems. In this paper, it is presented the so-called “Vigilance and Attention Controller”, a system based on electroencephalography (EEG) and eye-tracking (ET) techniques, aimed to assess in real time the vigilance level of an ATCo dealing with a highly automated human–machine interface and to use this measure to adapt the level of automation of the interface itself. The system has been tested on 14 professional ATCos performing two highly realistic scenarios, one with the system disabled and one with the system enabled. The results confirmed that (i) long high automated tasks induce vigilance decreasing and OOTL-related phenomena; (ii) EEG measures are sensitive to these kinds of mental impairments; and (iii) AA was able to counteract this negative effect by keeping the ATCo more involved within the operative task. The results were confirmed by EEG and ET measures as well as by performance and subjective ones, providing a clear example of potential applications and related benefits of AA.

Neurophysiological Vigilance Characterisation and Assessment: Laboratory and Realistic Validations Involving Professional Air Traffic Controllers

“…According to the most recent annual global statistics, provided by the International Civil Aviation Organization (ICAO) [41], the total number of passengers carried rose to 4.1 billion in 2017, 7.2% higher than the previous year, while the number of departures reached 36.7 million in 2017, a 3.1% increase compared to 2016. Furthermore, the 4.1 billion airline passengers carried in 2017 is expected to grow to about 10.0 billion by 2040, and the number of departures is projected to rise to some 90 million in 2040 [42]. So, in the aviation field, in particular, in Air Traffic Control (ATC), automation is recognised as an important topic.…”

“…The real traffic data of the International Munich Airport was used and its call-signs changed between the scenarios to avoid learning effects. The data were collected during a previous study that aimed at realising an online system to prevent out-of-the-loop phenomenon [42,68] which calibrated the "vigilance and attention controller" (VAC) [69] on each ATCO before the experiment. The VAC performance has already been investigated [42] and is out of the scope of the present manuscript.…”

“…The data were collected during a previous study that aimed at realising an online system to prevent out-of-the-loop phenomenon [42,68] which calibrated the "vigilance and attention controller" (VAC) [69] on each ATCO before the experiment. The VAC performance has already been investigated [42] and is out of the scope of the present manuscript. Three different scenarios of 45 min were realised: a training scenario (TRAINING) and two experimental ones (BASELINE and SOLUTION).…”

“…This future system architecture and automations level are seen as the way to envisage the future growth of air traffic while maintaining ATM safety. However, it is well known that prolonged supervisory activity impacts the controller's sustained attention, or vigilance, levels by significantly reducing the reaction time and increasing the probability of missing important information or identify risky events [7,8,42,78].…”

“…Experimental setup: Air traffic controller's brain activity has been recorded during the execution of the air traffic management scenarios. Source:[42]. The authors own the copyright of the picture.…”