Sensor Networks for Aerospace Human-Machine Systems

Abstract

Intelligent automation and trusted autonomy are being introduced in aerospace cyber-physical systems to support diverse tasks including data processing, decision-making, information sharing and mission execution. Due to the increasing level of integration/collaboration between humans and automation in these tasks, the operational performance of closed-loop human-machine systems can be enhanced when the machine monitors the operator’s cognitive states and adapts to them in order to maximise the effectiveness of the Human-Machine Interfaces and Interactions (HMI2). Technological developments have led to neurophysiological observations becoming a reliable methodology to evaluate the human operator’s states using a variety of wearable and remote sensors. The adoption of sensor networks can be seen as an evolution of this approach, as there are notable advantages if these sensors collect and exchange data in real-time, while their operation is controlled remotely and synchronised. This paper discusses recent advances in sensor networks for aerospace cyber-physical systems, focusing on Cognitive HMI2 (CHMI2) implementations. The key neurophysiological measurements used in this context and their relationship with the operator’s cognitive states are discussed. Suitable data analysis techniques based on machine learning and statistical inference are also presented, as these techniques allow processing both neurophysiological and operational data to obtain accurate cognitive state estimations. Lastly, to support the development of sensor networks for CHMI2 applications, the paper addresses the performance characterisation of various state-of-the-art sensors and the propagation of measurement uncertainties through a machine learning-based inference engine. Results show that a proper sensor selection and integration can support the implementation of effective human-machine systems for various challenging aerospace applications, including Air Traffic Management (ATM), commercial airliner Single-Pilot Operations (SIPO), one-to-many Unmanned Aircraft Systems (UAS), and space operations management.

Human Missions Analysis for Intelligent Missions Improvement

“…Intelligent automation and trusted autonomy are being introduced in aerospace cyber-physical systems to support diverse tasks including decision-making, data processing, information sharing, and mission execution with the technological developments of sensor networks [21]. This leads to the field of AI in manned space missions.…”

A Cyber-Physical-Human System for One-to-Many UAS Operations: Cognitive Load Analysis

“…The measurement of cognitive load, particularly MWL, in real-time gives CPS the ability to sense and adapt to the human operator. The proposed Cognitive Human Machine System (CHMS) is a CPH system concept that incorporates system automation support, which modulates as a function of measured cognitive state of the human operator [ 3 , 4 , 5 ]. Among other functions, the system allows dynamic adaptation of the system Automation Level (AL) and actual command/control interfaces, while maintaining desired MWL and the highest possible level of situational awareness.…”

“…For instance, Electroencephalogram (EEG) electrodes are prone to internal and external artifacts such as eye blinks, movement, heartbeat artifacts and other electromagnetic interference [ 9 ], whereas blink rate and pupillometry are among other sensitive to ambient light stimuli [ 10 , 11 ]. Henceforth, in a CHMS the monitoring of multiple parameters in a sensor network ensures the integrity of the system [ 3 ]. Such a sensor network is also natively suited to exploit data fusion of the physiological measurements to increase the overall accuracy and reliability of the human operators’ estimated MWL.…”

“…Nonetheless, the inference of cognitive states based on physiological data is still an active area of research, with the most promising avenue being the use of AI techniques including supervised Machine Learning (ML) to generate models of the users’ cognitive states based on labeled data [ 12 , 13 , 15 , 16 , 17 ]. For a more detailed review on the various physiological sensors, and the corresponding methods implemented for processing MWL measurements see the following reference [ 3 ].…”

“…In regard to gaze features, the scan path can allow for more complex features to be extracted such as visual entropy [ 45 ]. The eye tracking features correlated with the cognitive state include fixation, blink rate, saccades, pupil diameter, dwell time and visual entropy [ 3 ]. Visual entropy provides a particularly useful measure, where studies have shown that visual entropy was able to discriminate between control modes and flight phases associated with different levels of MWL [ 46 ].…”