No abstract

High risk professions, such as pilots, police officers, and TSA agents, require sustained vigilance over long periods of time and/or under conditions of little sleep. This can lead to performance impairment in occupational tasks. Predicting impaired states before performance decrement manifests is critical to prevent costly and damaging mistakes. We hypothesize that machine learning models developed to analyze indices of eye and face tracking technologies can accurately predict impaired states. To test this we trained 12 types of machine learning algorithms using five methods of feature selection with indices of eye and face tracking to predict the performance of individual subjects during a psychomotor vigilance task completed at 2-h intervals during a 25-h sleep deprivation protocol. Our results show that (1) indices of eye and face tracking are sensitive to physiological and behavioral changes concomitant with impairment; (2) methods of feature selection heavily influence classification performance of machine learning algorithms; and (3) machine learning models using indices of eye and face tracking can correctly predict whether an individual's performance is "normal" or "impaired" with an accuracy up to 81.6%. These methods can be used to develop machine learning based systems intended to prevent operational mishaps due to sleep deprivation by predicting operator impairment, using indices of eye and face tracking.

The "cocktail party effect" refers to the ability of human listeners to separate the acoustic signal reaching their ears into its individual components, corresponding to individual sound sources in the environment. Despite this phenomenon appearing trivial for humans, implementing the cocktail party effect computationally remains an ambitious challenge. The approach used in this paper takes inspiration from human strategies for separating an acoustic environment into distinct perceptual auditory streams. A series of time-frequency-based features, analogous to those thought to emerge at various stages in the human auditory processing pathway, are derived from biaural auditory inputs. These feature vectors are used as inputs to an unsupervised cluster analysis used to group feature values that are assumed to correspond to the same object. Reconstructed auditory streams are then correlated to the original components used to create the auditory scene. Our model is capable of reconstructing streams that correlate to the original components (r = 0.3-0.7) used to create the complex auditory scene. The success of the reconstructions is largely dependent on the signal-to-noise ratio of the components of the auditory scene.

No abstract

Introduction The Naval Submarine Medical Research Laboratory (NSMRL) is developing predictive models to examine how non-invasive, non-disruptive physiological monitoring can be used to track performance decrements due to sleep deficiency. Utilizing biometrics extracted from physiological measures to track performance changes would allow for automated tracking of fatigue and alleviate the overhead necessary to monitor individual schedules and sleep patterns. Methods NSMRL collaborated with the University of Connecticut to run a sleep deprivation study that deprived 20 participants of sleep for a period of up to 25 hours. During this time, subjects completed multiple tasks, including the Psychomotor Vigilance Test (PVT) every few hours. A non-invasive monitoring system collected physiological data from participants, which includes eye tracking, electrocardiography, electrodermal activity, and facial tracking (e.g., blink metrics, heart rate variability, skin conductance levels, facial action units). Using this multimodal approach, biometrics were extracted and evaluated to determine their predictive power on PVT performance. Multiple linear regression, using predictors selected via sequential forward selection, was used to develop a model of performance at an individual level based on a subset of these metrics chosen using principal component regression. Results Thirty-eight biometrics were extracted from the collected data and used to produce a predictive model of PVT performance. Sequential forward selection was used to select 11 primary biometrics. The criteria for primary metric inclusion in the model was minimization of root mean squared error. The resultant model had a correlation coefficient (r) of 0.71 (p < 0.001) with a root mean squared error (RMSE) of 49.8 ms between the predicted reaction time and true reaction time for each subject. Conclusion Non-invasive, non-disruptive monitoring could be used to track individual cognitive performance decrement due to sleep deficiency. This study examined the capability of combining the data from four physiological monitors that can be contained within a wrist worn device and a desk or helmet mounted camera. Utilizing 11 biometrics obtained from these monitors a stepwise regression model was developed that significantly correlates with PVT reaction time at both an individual and group level. Support This work was supported by the Military Operational Medicine Research Program.