Eye Tracking-Based Workload and Performance Assessment for Skill Acquisition

Mark, Jesse; Curtin, Adrian; Kraft, Amanda; Sands, Trevor; Casebeer, William D.; Ziegler, Matthias; Ayaz, Hasan

doi:10.1007/978-3-030-20473-0_14

Cited by 6 publications

(6 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To our surprise in this project, we found that a significant difference was only displayed in the maxAPS between easy and difficult tasks, not in the minAPS, meanAPS, medianAPS, and max interval index [ 22 ]. These are not duplicating results from previous studies [ 15 , 23 , 24 ]. A possible explanation is that most of the easy subtasks including identification of landmarks, disinfection, local anesthesia, and incision, are at the beginning of the procedure.…”

Section: Discussionsupporting

confidence: 65%

When medical trainees encountering a performance difficulty: evidence from pupillary responses

et al. 2022

View full text Add to dashboard Cite

Background Medical trainees are required to learn many procedures following instructions to improve their skills. This study aims to investigate the pupillary response of trainees when they encounter moment of performance difficulty (MPD) during skill learning. Detecting the moment of performance difficulty is essential for educators to assist trainees when they need it. Methods Eye motions were recorded while trainees practiced the thoracostomy procedure in the simulation model. To make pupillary data comparable among trainees, we proposed the adjusted pupil size (APS) normalizing pupil dilation for each trainee in their entire procedure. APS variables including APS, maxAPS, minAPS, meanAPS, medianAPS, and max interval indices were compared between easy and difficult subtasks; the APSs were compared among the three different performance situations, the moment of normal performance (MNP), MPD, and moment of seeking help (MSH). Results The mixed ANOVA revealed that the adjusted pupil size variables, such as the maxAPS, the minAPS, the meanAPS, and the medianAPS, had significant differences between performance situations. Compared to MPD and MNP, pupil size was reduced during MSH. Trainees displayed a smaller accumulative frequency of APS during difficult subtask when compared to easy subtasks. Conclusions Results from this project suggest that pupil responses can be a good behavioral indicator. This study is a part of our research aiming to create an artificial intelligent system for medical trainees with automatic detection of their performance difficulty and delivering instructional messages using augmented reality technology.

show abstract

Section: Discussionsupporting

confidence: 65%

When medical trainees encountering a performance difficulty: evidence from pupillary responses

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The last metric identified from the literature related to monitoring visual behavior is the first fixation duration, which means how long the first fixation lasts. Gaze Location [10,12,[22][23][24][25] Pupil Dilation [11] Fixation Duration [4,13,15,[18][19][20][21][22]26] Fixation Frequency [4,13,14,16,18,19,21,25] Saccade Duration [4,13,20] Saccade Frequency [14][15][16]19] Total Recording Duration [19,26] First Fixation Duration [15] 1.…”

Section: Application Of Eye-tracking Technologymentioning

confidence: 99%

“…They applied virtual reality technology to simulate the experiment environment and monitored the eye-movements of experiment participants using an eye-tracker to analyze and measure their ability in reasoning. Mark et al [ 11 ] did a combined psychological–neurological experiment on twenty-three subjects to assess their improvement during four weeks of training in a six-task cognitive battery. They measured the participants’ eye movement in addition to other psychological criteria such as heart rate as indicators of the subjects’ performance to prove that, by increasing in skill level, the workload related to vigilance skill could be increased over time while the starting workload was decreased.…”

Section: Introductionmentioning

confidence: 99%

Industrial Energy Assessment Training Effectiveness Evaluation: An Eye-Tracking Study

Ghanbari

Wang

Jeon

2021

Sensors

View full text Add to dashboard Cite

It is essential to understand the effectiveness of any training program so it can be improved accordingly. Various studies have applied standard metrics for the evaluation of visual behavior to recognize the areas of interest that attract individuals’ attention as there is a high correlation between attentional behavior and where one is focusing on. However, through reviewing the literature, we believe that studies that applied eye-tracking technologies for training purposes are still limited, especially in the industrial energy assessment training field. In this paper, the effectiveness of industrial energy assessment training was quantitatively evaluated by measuring the attentional allocation of trainees using eye-tracking technology. Moreover, this study identifies the areas that require more focus based on evaluating the performance of subjects after receiving the training. Additionally, this research was conducted in a controlled environment to remove the distractions that may be caused by environmental factors to only concentrate on variables that influence the learning behavior of subjects. The experiment results showed that after receiving the training, the subjects’ performance in energy assessment was significantly improved in two areas: production, and recycling and waste management, and the designed training program enhanced the knowledge of participants in identifying energy-saving opportunities to the knowledge level of experienced participants.

show abstract

“…Benfatto et al [ 14 ] apply eye-tracking features to detect psychological disorders. Workload and performance of skill acquisition based on eye-tracking data are presented in [ 15 ]. Eye-movement based classification of visual and linguistic tasks is discussed in [ 16 ].…”

Section: Introductionmentioning

confidence: 99%

Interpretable Machine Learning Models for Three-Way Classification of Cognitive Workload Levels for Eye-Tracking Features

2021

View full text Add to dashboard Cite

The paper is focussed on the assessment of cognitive workload level using selected machine learning models. In the study, eye-tracking data were gathered from 29 healthy volunteers during examination with three versions of the computerised version of the digit symbol substitution test (DSST). Understanding cognitive workload is of great importance in analysing human mental fatigue and the performance of intellectual tasks. It is also essential in the context of explanation of the brain cognitive process. Eight three-class classification machine learning models were constructed and analysed. Furthermore, the technique of interpretable machine learning model was applied to obtain the measures of feature importance and its contribution to the brain cognitive functions. The measures allowed improving the quality of classification, simultaneously lowering the number of applied features to six or eight, depending on the model. Moreover, the applied method of explainable machine learning provided valuable insights into understanding the process accompanying various levels of cognitive workload. The main classification performance metrics, such as F1, recall, precision, accuracy, and the area under the Receiver operating characteristic curve (ROC AUC) were used in order to assess the quality of classification quantitatively. The best result obtained on the complete feature set was as high as 0.95 (F1); however, feature importance interpretation allowed increasing the result up to 0.97 with only seven of 20 features applied.

show abstract

Eye Tracking-Based Workload and Performance Assessment for Skill Acquisition

Cited by 6 publications

References 33 publications

When medical trainees encountering a performance difficulty: evidence from pupillary responses

When medical trainees encountering a performance difficulty: evidence from pupillary responses

Industrial Energy Assessment Training Effectiveness Evaluation: An Eye-Tracking Study

Interpretable Machine Learning Models for Three-Way Classification of Cognitive Workload Levels for Eye-Tracking Features

Contact Info

Product

Resources

About