Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

Castillo-Segura, Pablo; Fernandez‐Panadero, Carmen; Alario-Hoyos, Carlos; Muñoz‐Merino, Pedro J.; Kloos, Carlos Delgado

doi:10.1016/j.artmed.2020.102007

Cited by 20 publications

(13 citation statements)

References 110 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In previous self tutoring frameworks, the proposed coaching was based on expert level classification or pre-recorded expert parameters such as videos, benchmarks, or milestones 9 , 33 – 35 . In contrast to determining feedback based on expertise group classification or static parameters, the ICEMS produces dynamic feedback for each performance metric by separate algorithms.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation

et al. 2022

View full text Add to dashboard Cite

In procedural-based medicine, the technical ability can be a critical determinant of patient outcomes. Psychomotor performance occurs in real-time, hence a continuous assessment is necessary to provide action-oriented feedback and error avoidance guidance. We outline a deep learning application, the Intelligent Continuous Expertise Monitoring System (ICEMS), to assess surgical bimanual performance at 0.2-s intervals. A long-short term memory network was built using neurosurgeon and student performance in 156 virtually simulated tumor resection tasks. Algorithm predictive ability was tested separately on 144 procedures by scoring the performance of neurosurgical trainees who are at different training stages. The ICEMS successfully differentiated between neurosurgeons, senior trainees, junior trainees, and students. Trainee average performance score correlated with the year of training in neurosurgery. Furthermore, coaching and risk assessment for critical metrics were demonstrated. This work presents a comprehensive technical skill monitoring system with predictive validation throughout surgical residency training, with the ability to detect errors.

show abstract

Section: Discussionmentioning

confidence: 99%

“…This work presents a technical skills continuous assessment application built using expert surgeon data, with predictive validity across a training program on surgical trainee performance 16 , 35 . This deep learning application demonstrated a granular differentiation across expertise and between resident levels.…”

Section: Discussionmentioning

confidence: 99%

Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation

et al. 2022

View full text Add to dashboard Cite

show abstract

“…While modern eye-tracking technologies are relatively inexpensive, userfriendly, unobtrusive and non-intrusive, there are still barriers for implementing eye-tracking in training and assessment reported in the literature. Several studies report costs as a barrier (Rosch & Vogel-Walcutt, 2013;Merali et al 2014;Castillo-Segura et al 2021), and several studies report challenges in taking them into use, including difficulties in selecting suitable tasks and/or uncertainty of how to measure specific skills (Fox & Faulkner-Jones, 2017;Merali et al 2017;Hermens et al 2013;Limbu et al 2018). Barriers related to difficulty in programming the equipment are also reported (Rosch & Vogel-Walcutt, 2013).…”

Section: Barriers and Benefits Reported In The Literaturementioning

confidence: 99%

“…Barriers related to difficulty in programming the equipment are also reported (Rosch & Vogel-Walcutt, 2013). Moreover, studies report that eye-tracking is a rather time-consuming method, generating large data sets that need to be properly analyzed (Merali et al 2017;Castillo-Segura et al 2021). Hence, using eyetracking technologies in training and assessment is still not an off-the-shelf pedagogical method, easily available for the everyday educational practices in complex socio-technical domains.…”

Section: Barriers and Benefits Reported In The Literaturementioning

confidence: 99%

Eye-tracking in simulator training and assessment: A semi-structured meta-review

Sellberg¹,

Praetorius²,

Nivala³

2022

Training, Education, and Learning Sciences

View full text Add to dashboard Cite

The potential for using eye-tracking as quantitative and objective tools for training and assessment has gained substantial attention across domain with high standards for safety [1]. The aim of this study is to synthesize results from previous reviews on eye-tracking support in training and assessment for work in complex socio-technical domains. In particular, the objective is to map out a) how eye-tracking is currently used in training and assessment, and b) the barriers and benefits of using eye-tracking reported in the literature. To answer the research questions, this study is designed as a meta-review, that is, a review of previous reviews or meta-analyses [2]. Since this study takes on broad re-search questions and aims to include different types of methodological contributions from various field to synthesize the state of knowledge between disciplines, a semi-structured review approach is chosen. Semi-structured approaches are suitable for mapping a field of research and synthesize the state of knowledge to set an agenda for future studies [3]. The approach takes on systematic search strategies but often lack the level of scientific rigour of systematic or scoping reviews. What is considered important for the semi-structured review method is a transparent and developed research strategy for the audi-ence to determine the worth and value of the chosen topic, method used and findings from the study [3]. In accordance with best practices, the study design is guided by the PRISMA-protocol for systematic reviews [4]. A search for literature was conducted in February 2021. Advanced search options for discriminating type of articles (reviews) and timespan (2010-2021) were used on three databases: Scopus, Web of Science and Science Direct. Using search words “eye tracking”, “simulator”, “training” and “assessment” when searching through article titles, abstracts and keywords, a total of 403 studies were identified. After removing duplicates, 373 studies were screened. Articles that didn’t meet the inclusion criteria or lacked an explicit focus on eye-tracking support in simulator training and/or assessment were excluded. In all, 20 full-text articles were assessed for eligibility. After reading full-text, 12 studies remained and are included in the meta-review. The results show how eye-tracking most commonly is used for 1) summative assessment of technical skills, 2) formative assessment of non-technical skills, 3) identifying differences between novices and experts, and 4) capturing expert gaze patterns. The results show how eye-tracking is appreciated as an unobtrusive technology that enables a quantitative assessment of objective parameters. Hence, the promise of using eye-tracking for informing instructional design as well as for designing training systems and improve learning environments is clear. However, it is important to consider that implementing eye-tracking is a rather expensive and time-consuming endeavour that requires carefully designed tasks and task analyses to fulfil its potential. References1.Rosch, J. L., & Vogel-Walcutt, J. J. A review of eye-tracking applications as tools for training. Cogn. Technol. Work. 15(3), 313–327 (2013)2.Sarrami‐Foroushani, P., Travaglia, J., Debono, D., Clay‐Williams, R., & Braithwaite, J. Scoping meta‐review: introducing a new methodology. Clin. Transl. Sci. 8(1), 77–81 (2015)3.Snyder, H. Literature review as a research methodology: An overview and guidelines. J. Bus. Res, 104, 333–339 (2019)4.Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., ... & Stewart, L. A. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst. Rev. 4(1), 1-9 (2015)

show abstract

“…During surgical training, the assessment of the technical surgical skills to be acquired by novice surgeons has traditionally been performed by expert surgeons, being a subjective assessment that may be biased. Recent advances in the Internet of Things (IoT), the ability to embed sensors in objects and environments to collect large amounts of data, and advances in machine learning allow for a more objective and automated assessment of MIS skills [5].…”

Section: Introductionmentioning

confidence: 99%

Wearable Technology for Assessment and Surgical Assistance in Minimally Invasive Surgery

Sánchez‐Margallo¹,

Rabazo²,

Miguel³

et al. 2022

Advances in Minimally Invasive Surgery

View full text Add to dashboard Cite

Wearable technology is an emerging field that has the potential to revolutionize healthcare. Advances in sensors, augmented reality devices, the internet of things, and artificial intelligence offer clinically relevant and promising functionalities in the field of surgery. Apart from its well-known benefits for the patient, minimally invasive surgery (MIS) is a technically demanding surgical discipline for the surgeon. In this regard, wearable technology has been used in various fields of application in MIS such as the assessment of the surgeon’s ergonomic conditions, interaction with the patient or the quality of surgical performance, as well as in providing tools for surgical planning and assistance during surgery. The aim of this chapter is to provide an overview based on the scientific literature and our experience regarding the use of wearable technology in MIS, both in experimental and clinical settings.

show abstract

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

Cited by 20 publications

References 110 publications

Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation

Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation

Eye-tracking in simulator training and assessment: A semi-structured meta-review

Wearable Technology for Assessment and Surgical Assistance in Minimally Invasive Surgery

Contact Info

Product

Resources

About