Optimal timing of assessment tasks depending on experience level of surgical trainees

Abstract

Introduction: Box trainers with motion analysis are important add-ons to surgical training and skills assessment outside the operating room, given that they exhibit construct validity. Material and Methods:Four different tasks were tested for construct validity on a new laparoscopic box trainer with integrated motion analysis. Tracking data from the simulator were analyzed for eighteen parameters per task using an in-house software comparing participants with three different experience levels.Results: In total, 10 novices, 22 intermediates and 16 experts enrolled. No or limited significant difference were found for the peg picker and rope race. For the precision cutting task 12 parameters showed significant difference between novices and intermediates, 14 between novices and experts and 1 between intermediates and experts. For the suture task the corresponding results were 1, 15 and 6. Conclusions:The precision cutting and suture task both showed construct validity for many of the parameters. While the precision cutting task distinguished best between novices and the other two groups, the suture task distinguished best between experts and the other two groups.These results show the importance of the timing of an assessment task, and that an assessment task might have limited value if experience levels are not considered.

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

“…However, their ergonomics can affect the surgeon's movement during the execution of the surgical procedure. Moreover, mechanical sensors have a limited range of movement due to their wired connection and are more difficult to move due to their size [28]. Electromagnetic (EM) sensors are also very common (24 articles) due to their reduced size and cost compared to mechanical sensors.…”

“…In summary, there are some research opportunities regarding the use of sensors for the objective and automated assessment of surgical technical skills. In relation to the movement of both instruments and surgeon's body, although there are a number of related publications, there are still research gaps on the reduction of the weight of sensors, the increase in the range of movements (for instruments and surgeons' body), and the creation of wireless IoT systems [28]; sensors can also be enhanced to have a higher number of Degrees of Freedom (DoF), in order to collect more data with one single sensor, as it is the case of Genovese et al [72], who used a sensor with nine DoF to collect three-axis orientation, as well as threeaxis linear and angular velocities. In relation to biometric signals aimed at representing the cognitive load of the surgeon, it is necessary to create IoT systems that allow the collection of more useful data, perhaps by combining multiple biometric signals at once [8].…”

“…Regarding the sensors used, there are some limitations which need to be addressed. First, it is necessary to improve ergonomics of the tools and sensors used to reach a wider range of motions [28], limited sometimes by the particular sensors used or by the need for wires to get power supply [71]. Second, it is necessary to use tools and sensors without problems with line of sight (in the case of e.g., optical sensors) [84], interferences with metallic objects (in the case of e.g., electromagnetic sensors) [71][65], or friction forces and involuntary vibrations (in the case of e.g., IMUs or force sensors) [54].…”

“…In contrast, fixation duration, fixation rate, fixation number, saccade number, saccade duration, and saccade amplitude did not show very good results to classify levels of expertise [105]. In addition, other indicators that are not derived from data collected by sensors can be included in classification models, such as years of experience [8] or years of study of the surgeon [25], GRS real score [47], OSATS real score [24], GEARS real score [36], number of procedures made [28], and time and number of errors made [86]. Regarding prediction models and the indicators used, indicators can be combined with some weights to obtain scores about surgical technical skills.…”

Construct Validity of a Novel Assessment System for Laparoscopic Suture Accuracy Based on Stereoscopy

“…Previous studies demonstrated that an operator's laparoscopic experience could be evaluated by task completion time and suture accuracy in laparoscopic suturing exercises [11, 12]. It was also found that the depth perception was different between laparoscopic experts and novices [13].…”