Evaluation of procedure-based assessment for assessing trainees' skills in the operating theatre

Marriott, J; Purdie, H; Crossley, James; Beard, Jonathan

doi:10.1002/bjs.7342

Cited by 95 publications

(105 citation statements)

References 10 publications

(11 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…36 Procedure-based assessment includes "exposure" and "technique." 37 And the Ottawa Surgical Competency Operating Room Score uses "knowledge of procedural steps." 38 But none of these rubrics fully capture the pattern of principles represented in this study.…”

Section: Acknowledging Challenges In Assessmentmentioning

confidence: 99%

“They Have to Adapt to Learn”: Surgeons’ Perspectives on the Role of Procedural Variation in Surgical Education

Apramian¹,

Cristancho²,

Watling³

et al. 2016

Journal of Surgical Education

View full text Add to dashboard Cite

OBJECTIVE-Clinical research increasingly acknowledges the existence of significant procedural variation in surgical practice. This study explored surgeons' perspectives regarding the influence of intersurgeon procedural variation on the teaching and learning of surgical residents.DESIGN AND SETTING-This qualitative study used a grounded theory-based analysis of observational and interview data. Observational data were collected in 3 tertiary care teaching hospitals in Ontario, Canada. Semistructured interviews explored potential procedural variations arising during the observations and prompts from an iteratively refined guide. Ongoing data analysis refined the theoretical framework and informed data collection strategies, as prescribed by the iterative nature of grounded theory research. RESULTS-Surgeons endorsed the use of intersurgeon procedural variations to teach residents about adapting to the complexity of surgical practice and the norms of surgical culture. Surgeons suggested that residents' efforts to identify thresholds of principle and preference are crucial to professional development. Principles that emerged from the study included the following: (1) knowing what comes next, (2) choosing the right plane, (3) handling tissue appropriately, (4) recognizing the abnormal, and (5) making safe progress. Surgeons suggested that learning to follow these principles while maintaining key aspects of surgical culture, like autonomy and individuality, are important social processes in surgical education. PARTICIPANTS-OurCONCLUSIONS-Acknowledging intersurgeon variation has important implications for curriculum development and workplace-based assessment in surgical education. Adapting to intersurgeon procedural variations may foster versatility in surgical residents. However, the existence of procedural variations and their active use in surgeons' teaching raises questions about the lack of attention to this form of complexity in current workplace-based assessment strategies. CIHR Author Manuscript CIHR Author Manuscript CIHR Author ManuscriptFailure to recognize the role of such variations may threaten the implementation of competencybased medical education in surgery.

show abstract

Section: Acknowledging Challenges In Assessmentmentioning

confidence: 99%

“They Have to Adapt to Learn”: Surgeons’ Perspectives on the Role of Procedural Variation in Surgical Education

Apramian¹,

Cristancho²,

Watling³

et al. 2016

Journal of Surgical Education

View full text Add to dashboard Cite

show abstract

“…Nonetheless, regardless of the method or technology used, if the simulation provides appropriate fidelity (faithfulness of reproduction) for key components of the task, it is likely that consistent and uniform skills may be learned and assessed using computer-based methods. This contrasts with the unpredictability and bias of patient-based training and assessment [17].…”

Section: What Can We Expect From the Technology?mentioning

confidence: 66%

Simulation: Moving from Technology Challenge to Human Factors Success

Gould

Chalmers

Johnson

et al. 2011

Cardiovasc Intervent Radiol

View full text Add to dashboard Cite

Recognition of the many limitations of traditional apprenticeship training is driving new approaches to learning medical procedural skills. Among simulation technologies and methods available today, computer-based systems are topical and bring the benefits of automated, repeatable, and reliable performance assessments. Human factors research is central to simulator model development that is relevant to real-world imaging-guided interventional tasks and to the credentialing programs in which it would be used.

show abstract

“…17 PBA assessments are now a well-established, validated method for trainee assessment. 18,19 These assessments have been shown in prospective studies to have good construct validity, which is mandatory for a competency evaluator, and excellent reliability, while being superior to other assessment tools. 18,20,21 It has been suggested that WBA assessments should continue throughout training, at an approximate rate of one per week.…”

Section: Discussionmentioning

confidence: 99%

“…18,19 These assessments have been shown in prospective studies to have good construct validity, which is mandatory for a competency evaluator, and excellent reliability, while being superior to other assessment tools. 18,20,21 It has been suggested that WBA assessments should continue throughout training, at an approximate rate of one per week. 13 A total of 76 WBAs relating to the four index procedures (61 PBAs, 15 CBDs) were undertaken during the EGS blocks, at approximately seven per month.…”

Section: Discussionmentioning

confidence: 99%

Modular emergency general surgery training: A pilot study of a novel programme

Egan¹,

Abdelrahman²,

Tate³

et al. 2016

annals

View full text Add to dashboard Cite

INTRODUCTION Pan-speciality consensus guidance advocates mandatory emergency general surgery (EGS) training modules for specialist registrars (StRs). This pilot study evaluated the impact of EGS modules aimed at StRs over 1 year. METHODS Eleven StRs were allocated a focused 4-week EGS module, in addition to the standard 1:12 on-call duty rota, in a tertiary surgical centre. Primary outcome measures included the number of indicative emergency operations and validated Procedure Based Assessments (PBAs) performed, both during the EGS module and over the training year. RESULTS StRs performed a median of 11 (range 5-15) laparotomies during the EGS module versus 31 (range 9-49) over the whole training year. StRs attended 43.7% of available laparotomies during the module (range 24.1-63.7%). EGS modules provided more than one-third of the total emergency laparotomy experience, and a quarter of the emergency colectomy, appendicectomy and Hartmann's procedure experience. There were no differences in EGS module-related outcomes between junior and senior StRs. Significantly more PBAs related to laparotomy and segmental colectomy were completed during EGS modules than the oncall duty rota, at 32% versus 14% (p<0.001) and 48% versus 22% (p=0.019), respectively. Performance levels were maintained following module completion. CONCLUSIONS These findings provide an important baseline when considering future modular EGS training.

show abstract

Evaluation of procedure-based assessment for assessing trainees' skills in the operating theatre

Abstract: PBA demonstrated good overall validity and acceptability, and exceptionally high reliability. Trainees should be assessed adequately for each given procedure.

Cited by 95 publications

References 10 publications

“They Have to Adapt to Learn”: Surgeons’ Perspectives on the Role of Procedural Variation in Surgical Education

“They Have to Adapt to Learn”: Surgeons’ Perspectives on the Role of Procedural Variation in Surgical Education

Simulation: Moving from Technology Challenge to Human Factors Success

Modular emergency general surgery training: A pilot study of a novel programme

Contact Info

Product

Resources

About