Eye-tracking in simulator training and assessment: A semi-structured meta-review

Abstract

The potential for using eye-tracking as quantitative and objective tools for training and assessment has gained substantial attention across domain with high standards for safety [1]. The aim of this study is to synthesize results from previous reviews on eye-tracking support in training and assessment for work in complex socio-technical domains. In particular, the objective is to map out a) how eye-tracking is currently used in training and assessment, and b) the barriers and benefits of using eye-tracking reported in the literature. To answer the research questions, this study is designed as a meta-review, that is, a review of previous reviews or meta-analyses [2]. Since this study takes on broad re-search questions and aims to include different types of methodological contributions from various field to synthesize the state of knowledge between disciplines, a semi-structured review approach is chosen. Semi-structured approaches are suitable for mapping a field of research and synthesize the state of knowledge to set an agenda for future studies [3]. The approach takes on systematic search strategies but often lack the level of scientific rigour of systematic or scoping reviews. What is considered important for the semi-structured review method is a transparent and developed research strategy for the audi-ence to determine the worth and value of the chosen topic, method used and findings from the study [3]. In accordance with best practices, the study design is guided by the PRISMA-protocol for systematic reviews [4]. A search for literature was conducted in February 2021. Advanced search options for discriminating type of articles (reviews) and timespan (2010-2021) were used on three databases: Scopus, Web of Science and Science Direct. Using search words “eye tracking”, “simulator”, “training” and “assessment” when searching through article titles, abstracts and keywords, a total of 403 studies were identified. After removing duplicates, 373 studies were screened. Articles that didn’t meet the inclusion criteria or lacked an explicit focus on eye-tracking support in simulator training and/or assessment were excluded. In all, 20 full-text articles were assessed for eligibility. After reading full-text, 12 studies remained and are included in the meta-review. The results show how eye-tracking most commonly is used for 1) summative assessment of technical skills, 2) formative assessment of non-technical skills, 3) identifying differences between novices and experts, and 4) capturing expert gaze patterns. The results show how eye-tracking is appreciated as an unobtrusive technology that enables a quantitative assessment of objective parameters. Hence, the promise of using eye-tracking for informing instructional design as well as for designing training systems and improve learning environments is clear. However, it is important to consider that implementing eye-tracking is a rather expensive and time-consuming endeavour that requires carefully designed tasks and task analyses to fulfil its potential. References1.Rosch, J. L., & Vogel-Walcutt, J. J. A review of eye-tracking applications as tools for training. Cogn. Technol. Work. 15(3), 313–327 (2013)2.Sarrami‐Foroushani, P., Travaglia, J., Debono, D., Clay‐Williams, R., & Braithwaite, J. Scoping meta‐review: introducing a new methodology. Clin. Transl. Sci. 8(1), 77–81 (2015)3.Snyder, H. Literature review as a research methodology: An overview and guidelines. J. Bus. Res, 104, 333–339 (2019)4.Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., ... & Stewart, L. A. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst. Rev. 4(1), 1-9 (2015)