Classification of pilots’ mental states using a multimodal deep learning network

Han, Sang-oh; Kwak, No Sang; Oh, Taegeun; Lee, Seong Whan

doi:10.1016/j.bbe.2019.12.002

Cited by 72 publications

(34 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Input: (1) For h ← 1to Hdo (2) A bagged subset of samples is drawn to F h from F (3) While (failure of stopping criteria) do (4) Random selection of m sub features are done (5) For m ← 1 to kmsubkdo (6) Reduction in the node input is computed (7) The feature which mitigates the inputs to the utmost level is chosen and then the node is divided into 2 children nodes. (8) The Htrees are combined to form a random forest.…”

Section: Algorithm 3 Random Forest Classification With Bootstrap Resampling Techniquementioning

confidence: 99%

“…For evaluating the various mental disorders like screening and diagnosis, EEG signals are predominantly utilized. By nature, EEG signals are highly dynamic and non-linear in nature and so it is utilized in various tasks such as classification of pilot mental states [ 8 ], drowsiness level detection [ 9 ], epilepsy detection [ 10 ], autism disorder detection [ 11 ], Alzheimer's disease detection [ 12 ], stroke analysis [ 13 ], consciousness and unconsciousness analysis [ 14 ], motor imagery classification [ 15 ], sleep related disorders [ 16 ], schizophrenia related disorders [ 17 ] etc. A more precise work for alcoholic EEG signal classification was done by Acharya et al in [ 18 ], where the alcoholic EEG signals was split from normal EEG signals by using non-linear features and Higher Order Spectra (HOS) features.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Alcoholic EEG signal classification with Correlation Dimension based distance metrics approach and Modified Adaboost classification

Prabhakar

Rajaguru²

2020

Heliyon

View full text Add to dashboard Cite

The basic function of the brain is severely affected by alcoholism. For the easy depiction and assessment of the mental condition of a human brain, Electroencephalography (EEG) signals are highly useful as it can record and measure the electrical activities of the brain much to the satisfaction of doctors and researchers. Utilizing the standard conventional techniques is quite hectic to derive the useful information as these signals are highly non-linear and non-stationary in nature. While recording the EEG signals, the activities of the neurons are recorded from various scalp regions which has varied characteristics and has a very low magnitude. Therefore, human interpretation of such signals is very difficult and consumes a lot of time. Hence, with the advent of Computer Aided Diagnosis (CAD) Techniques, identifying the normal versus alcoholic EEG signals has been of great utility in the medical field. In this work, we perform the initial clustering of the alcoholic EEG signals by means of using Correlation Dimension (CD) for easy feature extraction and then the suitable features are selected in it by means of employing various distance metrics like correlation distance, city block distance, cosine distance and chebyshev distance. Proceeding in such a methodology aids and assures that a good discrimination could be achieved between normal and alcoholic EEG signals using non-linear features. Finally, classification is then carried out with the suitable classifiers chosen such as Adaboost.RT classifier, the proposed Modified Adaboost.RT classifier by means of introducing Ridge and Lasso based soft thresholding technique, Random Forest with bootstrap resampling technique, Artificial Neural Networks (ANN) such as Radial Basis Functions (RBF) and Multi-Layer Perceptron (MLP), Support Vector Machine (SVM) with Linear, Polynomial and RBF Kernel, Naïve Bayesian Classifier (NBC), K-means classifier, and K Nearest Neighbor (KNN) Classifier and the results are analyzed. Results report a comparatively high classification accuracy of about 98.99% when correlation distance metrics are utilized with CD and the proposed Modified Adaboost.RT classifier using Ridge based soft thresholding technique.

show abstract

Section: Algorithm 3 Random Forest Classification With Bootstrap Resampling Techniquementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Alcoholic EEG signal classification with Correlation Dimension based distance metrics approach and Modified Adaboost classification

Prabhakar

Rajaguru²

2020

Heliyon

View full text Add to dashboard Cite

show abstract

“…(2019) evaluated the pilot's seat in the cockpit of the aircraft. Han et al (2020) utilized deep learning networks to evaluate the cognitive abilities of the pilot. Moreover, Brezonakova et al (2019) evaluated the pilot's visual fatigue.…”

Section: Related Workmentioning

confidence: 99%

Evaluating aircraft cockpit emotion through a neural network approach

Chen

Chu

et al. 2020

AIEDAM

View full text Add to dashboard Cite

Studies show that there are shortcomings in applying conventional methods for the emotional evaluation of the aircraft cockpit. In order to resolve this problem, a more efficient cockpit emotion evaluation system is established in the present study to simply and quickly obtain the cockpit emotion evaluation value. To this end, the neural network is applied to construct an emotional model to evaluate the emotional prediction of the interior design of the aircraft cockpit. Moreover, several technologies and the Kansei engineering method are applied to acquire the cockpit interior emotional evaluation data for typical aircraft models. In this regard, the radical basis function neural network (RBFNN), Elman neural network (ENN), and the general regression neural network (GRNN) are applied to construct the sentimental prediction evaluation model. Then, the three models are comprehensively compared through factors such as the model evaluation criteria, network structure, and network parameters. Obtained experimental results indicate that the GRNN not only has the highest classification accuracy but also has the highest stability in comparison to the other two neural networks, so that it is a more appropriate method for the emotional evaluation of the aircraft cockpit. Results of the present study provide decision supports for the emotional evaluation of the cockpit interior space.

show abstract

“…Recently, a number of papers have been published regarding the use of physiological sensors and deep learning for assessing MF, drowsiness, and mental workload [38][39][40][41]. Although some of these previous works present solid results, they usually also present a complex neural network structure, containing up to several million parameters, and computationally expensive algorithms.…”

Section: Mental Fatigue Assessmentmentioning

confidence: 99%

Investigating an Integrated Sensor Fusion System for Mental Fatigue Assessment for Demanding Maritime Operations

Monteiro

Skourup

et al. 2020

Sensors

View full text Add to dashboard Cite

Human-related issues are currently the most significant factor in maritime causalities, especially in demanding operations that require coordination between two or more vessels and/or other maritime structures. Some of these human-related issues include incorrect, incomplete, or nonexistent following of procedures; lack of situational awareness; and physical or mental fatigue. Among these, mental fatigue is especially dangerous, due to its capacity to reduce reaction time, interfere in the decision-making process, and affect situational awareness. Mental fatigue is also especially hard to identify and quantify. Self-assessment of mental fatigue may not be reliable and few studies have assessed mental fatigue in maritime operations, especially in real time. In this work we propose an integrated sensor fusion system for mental fatigue assessment using physiological sensors and convolutional neural networks. We show, by using a simulated navigation experiment, how data from different sensors can be fused into a robust mental fatigue assessment tool, capable of achieving up to 100 % detection accuracy for single-subject classification. Additionally, the use of different sensors seems to favor the representation of the transition between mental fatigue states.

show abstract

Classification of pilots’ mental states using a multimodal deep learning network

Cited by 72 publications

References 53 publications

Alcoholic EEG signal classification with Correlation Dimension based distance metrics approach and Modified Adaboost classification

Alcoholic EEG signal classification with Correlation Dimension based distance metrics approach and Modified Adaboost classification

Evaluating aircraft cockpit emotion through a neural network approach

Investigating an Integrated Sensor Fusion System for Mental Fatigue Assessment for Demanding Maritime Operations

Contact Info

Product

Resources

About