Utility of digitally supported surgical competency assessments in a work‐based setting: a systematic review of the literature

Abstract

BackgroundAdequate training and assessment of competency in surgical trainees are important in minimizing adverse events and achieving good patient outcomes. The purpose was to investigate the current literature on digitally supported methods of facilitating work‐based competency assessment of surgical trainees.MethodsTwo concurrent literature searches were performed by two independent researchers using three databases: MEDLINE, Embase and Education Resources Information Center. Eligible studies reported on digitally supported tools for assessing surgical competency in a work‐based setting for interns, residents or trainees of a post‐graduate surgical programme. Studies focusing primarily on training courses, simulations or other tools for assessing surgical skills in a setting outside of work were excluded. Articles published as abstracts only, articles not published in English and review articles that did not contain original data were excluded.ResultsA total of 11 eligible studies were included, with six of these implementing smartphone application‐based programmes, and five utilizing web‐based programmes. Five studies implemented the ‘System for Improving and Measuring Procedural Learning’ smartphone application. Studies were based predominantly on general surgery residents, and were limited to the North American context. There was significant variability between studies regarding methodology, including the scoring system used to assess competency.ConclusionThis review confirms that digitally supported competency assessment of surgical trainees in a work‐based setting is both feasible and effective. Digital platforms allow evaluations to be performed in a timely fashion without significant disturbance to workflow.

A quantitative analysis of the perceived barriers to implementation of multisource feedback in surgical training

“…This was reflected in the scores provided by both supervisors and trainees (Table 1). The electronic delivery method and formatting of our pilot study was able to overcome this barrier 10,22 . In the original pilot study of the novel instrument, trainees on average took 14 min 36 s to complete the assessment, while supervisors took 26 min 56 s (84% longer) 10 .…”

“…The electronic delivery method and formatting of our pilot study was able to overcome this barrier. 10,22 In the original pilot study of the novel instrument, trainees on average took 14 min 36 s to complete the assessment, while supervisors took 26 min 56 s (84% longer). 10 Given the time commitment needed to complete evaluations, it is important to consider whether protected time should be provided to trainees and supervisors when this instrument is no longer research and formally used as part of an assessment program in surgical units.…”

“…In this follow-up study to our original pilot, a list of variables that impact multisource feedback assessments was created based on existing literature from international, and non-surgical contexts. 17,[21][22][23] The factors thought to be relevant to the provision and reception of accurate evaluations in the Australian surgical context were reviewed by four independent senior surgical and medical educators in the ACT Health and included in the final questionnaire with 15 items. Questions designed were single-step, delivered in a balance of affirmative, negative and neutral with no leading questions.…”

Entrustable professional activities in postgraduate general surgery training: a scoping review protocol

“…2 Although reviews related to general surgery have been published on specific aspects of CBME, for example, WBA, none have included papers describing strategies based on EPAs. [4][5][6][7][8][9][10][11] Some reviews have looked at EPA use in broader groups, for example, unspecified graduate medical education; however, these reviews contain few studies from surgical disciplines. [12][13][14] Therefore, there is a paucity of published guidance available for surgeoneducators to facilitate the planning and implementation of CBME, using an EPA framework.…”

“…Although reviews related to general surgery have been published on specific aspects of CBME, for example, WBA, none have included papers describing strategies based on EPAs 4–11. Some reviews have looked at EPA use in broader groups, for example, unspecified graduate medical education; however, these reviews contain few studies from surgical disciplines 12–14.…”