Predicting aviation training performance with multimodal affective inferences

Abstract

Affect influences learning and training through various cognitive, psychomotor and motivational processes. This research aims to examine the role of affect in aviation training. Participants’ (N = 19) affect and performance were examined in simulated aviation training while they performed ten tasks. Affective states were inferred from electrodermal activity, facial expression and NASA Taskload Index. Performance accuracy was graded with the rubrics provided by pilot instructors in CAE Inc. We found that arousal (inferred from electrodermal activity) positively predicted performance in the level 2 (easy) task (F(1, 17) = 7.408, p < 0.05, std β = 0.55). Mental workload (as measured from self‐report) negatively predicted performance in the level 3 (medium difficulty) (F(1, 15) = 4.598, p < 0.05, std β = −0.54) and level 4 (difficult) tasks (F(1, 15) = 12.85, p < 0.01, std β = −0.73), controlling for affect valence and arousal. This research is a preliminary step to a reconsideration of affect in theoretical frameworks in aviation. It demonstrates a comprehensive assessment of affect in aviation training, which could provide guidelines for instructional interventions to improve the overall training experience and pilot performance.

Detecting Emotions through Electrodermal Activity in Learning Contexts: A Systematic Review

“…Six studies used both tonic and phasic components to further analyze the EDA signal [ 52 , 53 , 56 , 58 , 61 , 69 ]. Additionally, six studies used tonic components, i.e., skin conductance level (SCL) only [ 39 , 40 , 49 , 55 , 63 , 65 ], and seven studies used phasic components, i.e., skin conductance response (SCR) only [ 33 , 36 , 50 , 60 , 64 , 66 , 68 ]. Six studies used the raw EDA signal for feature extraction and analysis, and two studies provided no information on processing [ 54 , 62 ].…”

“…Eight studies cleaned their signal, which implies the exclusion of movement from the physiological signal. Three of them used both filtering as well as cleaning [ 50 , 59 , 63 ]. A prevalent form of cleaning is manual and visual detection of movement artifacts [ 55 , 64 ].…”

“…As shown in Table 3 , descriptive features such as the mean, standard deviation, minimum, maximum, and range were extracted in 19 of the 27 studies. The most commonly used feature ( n = 18) is the mean EDA of an individual participant [ 36 , 39 , 40 , 46 , 49 , 50 , 52 , 53 , 54 , 56 , 57 , 58 , 59 , 63 , 65 , 67 , 69 , 70 , 71 ]. The mean EDA is the average of EDA in a specific time period, but how this is calculated differs between studies.…”

“…As mentioned above, the EDA signal consists of two components, which are phasic (SCR) and tonic (SCL). Several researchers focused specifically on the skin conductance responses (SCR) and calculated the average SCR during the entire learning session [ 36 ] or a specific phase in the learning process (such as a predefined task) [ 50 , 69 ]. Others focused on SCL and calculated the average of SCL of the whole learning session [ 56 , 65 ] or the average of SCL of a specific phase in the learning process (such as a predefined task) [ 40 , 49 , 65 ].…”

“…For example, self-report data can be modified by participants due to the awareness of their environment [ 46 ]. Measuring emotion from a single modality can result in partial inferences by overlooking other, more subconscious aspects of emotion [ 50 ]. Therefore, the second objective was extended to provide insights into how physiological arousal as measured by EDA varies during the learning process (unimodal) and to identify combinations of physiological arousal as measured by EDA with multimodal data streams to understand learning processes (multimodal).…”