Pilots’ mental workload prediction based on timeline analysis

Abstract

BACKGROUND: The aircraft cockpit is a highly intensive human-computer interaction system, and its design directly affects flight safety. OBJECTIVE: To optimize the display interface design in complex flight tasks, the present study aimed to propose a dynamic conceptual framework and a timeline task analysis method for the quantization of the dynamic time effect of mental workload and the influencing factors of task types in the mental workload prediction model. METHODS: The multi-factor mental workload prediction model based on attention resource allocation was integrated to establish the dynamic prediction model of mental workload. The ergonomics simulation experiment was carried out by recording the data on the performance of embedded subtasks, National Aeronautics and Space Administration-Task Load Index (NASA-TLX) subjective evaluation, and eye tracking. RESULTS: The results indicated that the prediction model had a good prediction accuracy and effectiveness under different simulated interfaces and complex tasks, and the real-time monitoring of pilots’ mental workload state was realized. CONCLUSION: In conclusion, the prediction model and the experimental method could be applied to avoid the overload of the pilot throughout the flight phase by optimizing the display interface and adjusting the flight task.

Mental workload evaluation model of receiver aircraft pilots based on multiple resource theory

“…Time occupation is the time occupied by each action unit in the visual, auditory, cognitive, and psychomotor channels. According to the timeline analysis (Liu et al, 2020), the time from the perception of external information to the final execution of the operation includes reaction time t c and movement time t m . The reaction time t c can be further divided into simple reaction time t x and selected reaction time t s .…”

“…The reaction time t c can be further divided into simple reaction time t x and selected reaction time t s . The simple reaction time t x of each channel for the RA pilot to execute the task can usually be calculated by referring to the empirical data (Liu et al, 2020) and t m using Hicks' law (Liu et al, 2020;Schuck, 1996), which is expressed by the below equation.…”

“…The movement time was calculated by Fitts' law (Liu et al, 2020;Schuck, 1996) and expressed by the below equation.…”

“…Time occupation is the time occupied by each action unit in the visual, auditory, cognitive, and psychomotor channels. According to the timeline analysis (Liu et al, 2020), the time from the perception of external information to the final execution of the operation includes reaction time $<math altimg="urn:x-wiley:10908471:media:hfm21018:hfm21018-math-0005" wiley:location="equation/hfm21018-math-0005.png" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><msub><mi>t</mi><mi mathvariant="normal">c</mi></msub></mrow></mrow></math>$ and movement time $<math altimg="urn:x-wiley:10908471:media:hfm21018:hfm21018-math-0006" wiley:location="equation/hfm21018-math-0006.png" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><msub><mi>t</mi><mi>m</mi></msub></mrow></mrow></math>$. The reaction time $<math altimg="urn:x-wiley:10908471:media:hfm21018:hfm21018-math-0007" wiley:location="equation/hfm21018-math-0007.png" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><msub><mi>t</mi><mi mathvariant="normal">c</mi></msub></mrow></mrow></math>$ can be further divided into simple reaction time $<math altimg="urn:x-wiley:10908471:media:hfm21018:hfm21018-math-0008" wiley:location="equation/hfm21018-math-0008.png" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><msub><mi>t</mi><mi>x</mi></msub></mrow></mrow></math>$ and selected reaction time $<math altimg="urn:x-wiley:10908471:media:hfm21018:hfm21018-math-0009" wiley:location="equation/hfm21018-math-0009.png" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><msub><mi>t</mi><mi>s</mi></msub></mrow></mrow></math>$.…”

“…The reaction time $<math altimg="urn:x-wiley:10908471:media:hfm21018:hfm21018-math-0007" wiley:location="equation/hfm21018-math-0007.png" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><msub><mi>t</mi><mi mathvariant="normal">c</mi></msub></mrow></mrow></math>$ can be further divided into simple reaction time $<math altimg="urn:x-wiley:10908471:media:hfm21018:hfm21018-math-0008" wiley:location="equation/hfm21018-math-0008.png" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><msub><mi>t</mi><mi>x</mi></msub></mrow></mrow></math>$ and selected reaction time $<math altimg="urn:x-wiley:10908471:media:hfm21018:hfm21018-math-0009" wiley:location="equation/hfm21018-math-0009.png" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><msub><mi>t</mi><mi>s</mi></msub></mrow></mrow></math>$. The simple reaction time $<math altimg="urn:x-wiley:10908471:media:hfm21018:hfm21018-math-0010" wiley:location="equation/hfm21018-math-0010.png" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><msub><mi>t</mi><mi>x</mi></msub></mrow></mrow></math>$ of each channel for the RA pilot to execute the task can usually be calculated by referring to the empirical data (Liu et al, 2020) and $<math altimg="urn:x-wiley:10908471:media:hfm21018:hfm21018-math-0011" wiley:location="equation/hfm21018-math-0011.png" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><msub><mi>t</mi><mi>m</mi></msub></mrow></mrow></math>$ using Hicks' law (Liu et al, 2020; Schuck, 1996), which is expressed by the below equation. $$<math altimg="urn:x-wiley:10908471:media:hfm21018:hfm21018-math-0012" display="block" wiley:location="equation/hfm21018-math-0012.png" xmlns="http://www.w3.org/1998/Math/MathML"><mrow><mrow><msub><mi>t</mi><mi>s</mi></msub><mo>\unicode{x0003D}</mo><msub><mi>i</mi><mi>s</mi></msub><mo>\unicode{x000D7}</mo><mi>h</mi><mo>,</mo></mrow></mrow></math>$$where, $<math altimg="urn:x-w...$…”