Centralized Assessment in Graduate Medical Education: Cents and Sensibilities

Wagner, Dianne; Lypson, Monica L.

doi:10.4300/01.01.0004

Cited by 22 publications

(19 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…case is that the reliability, stability and relevance of these measures have not been established [11][12][13] creating lingering questions about the utility of the evaluation system to support a pathway to accreditation [7,15].…”

Section: Robotic Minimally Invasive Surgical Skill Assessment Based Omentioning

confidence: 99%

Robotic Minimally Invasive Surgical skill assessment based on automated video-analysis motion studies

Jun

Narayanan

Agarwal

et al. 2012

2012 4th IEEE RAS &Amp; EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob)

View full text Add to dashboard Cite

Abstract²Assessment of surgical skill, arising from the synthesis of the cognitive and sensorimotor capabilities of the surgeon, has predominantly been a subjective task. Development of quantitative metrics-of-performance with clinical relevance and other desirable characteristics (repeatability and stability) has always lagged behind. New opportunities for objective and automated assessment frameworks have arisen by virtue of technological advances in computation, video-processing, and data-acquisition, especially in the robotic Minimally Invasive Surgical (rMIS) realm. Most efforts focus on semi-quantitative (Likert scale) or inadequately validated, spatially-or temporally-aggregated quantitative metrics derived from direct physical measurements. In this work we propose an automated surgical expertise evaluation method, by adapting well-established motion studies methodologies, especially for MIS evaluation. This method relies on segmenting a primary task into sub-tasks, which can be evaluated by statistical analyses of micromotions. Motion studies were developed by 2 methods: (A) manual annotation process by experts (to serve as a benchmark); and (B) automatedkinematic-analysis-of-videos; for economy, repeatability as well as dexterity. The da Vinci SKILLS simulator was used to serve as a uniform testbed. Surgeons with varied levels of expertise were recruited to perform two representative simplified tasks (Peg Board and Pick & Place). The automated kinematic analysis of video was compared with the ground truth data (obtained by manual labeling) using misclassification rate and true classification confusion matrix. Future studies aimed towards analyzing real surgical procedures are already underway.

show abstract

Section: Robotic Minimally Invasive Surgical Skill Assessment Based Omentioning

confidence: 99%

Robotic Minimally Invasive Surgical skill assessment based on automated video-analysis motion studies

Jun

Narayanan

Agarwal

et al. 2012

2012 4th IEEE RAS &Amp; EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob)

View full text Add to dashboard Cite

show abstract

“…While few objective studies have compared the capabilities of current medical school graduates with the demands of current practice, local experience that new residents feel less than optimally prepared has given rise to ''academic boot camps'' to prepare the residents for the basic elements of supervised clinical practice. [12][13][14][15] Through the use of simulation, some of these ''boot camps'' have found deficiencies in basic skills, such as aseptic technique, hand hygiene, crosscultural communication, obtaining informed consent, 12 and basic procedure skills such as laceration management. 14 In surgical specialties, transitioning from medical school to residency has focused on the development of basic surgical technique.…”

Section: Transitions Across the Medical Education Continuummentioning

confidence: 99%

Review article: Closing the research gap at the interface of learning and clinical practice

Philibert

2011

Can J Anesth/J Can Anesth

View full text Add to dashboard Cite

Purpose The dialogue at the interface of education and clinical practice highlights areas of critical importance to the development of new approaches for educating anesthesiologists. The purpose of this article is to examine the literature on education and acquisition of competence in three areas relevant to the interface of learning and clinical practice, with the aim to suggest a research agenda that adds to the evidence on preparing physicians for independent practice. The three areas are: 1) transitions across the continuum of education; 2) the effect of reductions in hours of clinical training on competence; and 3) efforts to incorporate the competencies and CanMEDS roles into teaching and evaluation. Principal findings Fifty-six articles relevant to one or more of the themes were identified in the review, including 21 studies of transitions (in, during, and after residency education), 19 studies on the effects of duty hour limits on residents' acquisition of competence, and 16 articles that assessed competency-based teaching and assessment in anesthesiology. Overall, the findings suggested a relative paucity of scientific evidence and a need for research and the development of new scientific theory. Studies generally treated one of the themes in isolation, while in actuality they interact to produce optimal as well as suboptimal learning situations, while medical education research often is limited by small samples, brief follow-up, and threats to validity. This suggests a ''research gap'' where editorials and commentaries have moved ahead of an evidence base for education. Promising areas for research include preparation for care deemed important by society, work to apply knowledge about the development of expertise in other disciplines to medicine, and ways to embed the competencies in teaching and evaluation more effectively. Conclusion Closing the research gap in medical education will require clear direction for future work. The starting point, at an institution or nationally, is dialogue within the specialty to achieve consensus on some of the most pressing questions.

show abstract

“…[2][3][4][5][6] The heterogeneous nature of residents' on-the-job training 7 necessitates standardized assessment tools that mitigate inherent rater bias. 8,9 Reliable evaluation tools are essential for Milestone assessment in the Next Accreditation System for graduate medical education.…”

Section: Introductionmentioning

confidence: 99%

Test of Integrated Professional Skills: Objective Structured Clinical Examination/Simulation Hybrid Assessment of Obstetrics-Gynecology Residents' Skill Integration

Winkel¹,

Gillespie²,

Hiruma³

et al. 2014

Journal of Graduate Medical Education

View full text Add to dashboard Cite

Background Assessment of obstetrics-gynecology residents' ability to integrate clinical judgment, interpersonal skills, and technical ability in a uniform fashion is required to document achievement of benchmarks of competency. An observed structured clinical examination that incorporates simulation and bench models uses direct observation of performance to generate formative feedback and standardized evaluation. Methods The Test of Integrated Professional Skills (TIPS) is a 5-station performance-based assessment that uses standardized patients and complex scenarios involving ultrasonography, procedural skills, and evidence-based medicine. Standardized patients and faculty rated residents by using behaviorally anchored checklists. Mean scores reflecting performance in TIPS were compared across competency domains and by developmental level (using analysis of variance) and then compared to standard faculty clinical evaluations (using Spearman ρ). Participating faculty and residents were also asked to evaluate the usefulness of the TIPS. Results Twenty-four residents participated in the TIPS. Checklist items used to assess competency were sufficiently reliable, with Cronbach α estimates from 0.69 to 0.82. Performance improved with level of training, with wide variation in performance. Standard faculty evaluations did not correlate with TIPS performance. Several residents who were rated as average or above average by faculty performed poorly on the TIPS (> 1 SD below the mean). Both faculty and residents found the TIPS format useful, providing meaningful evaluation and opportunity for feedback. Conclusions A simulation-based observed structured clinical examination facilitates observation of a range of skills, including competencies that are difficult to observe and measure in a standardized way. Debriefing with faculty provides an important interface for identification of performance gaps and individualization of learning plans.

show abstract

Centralized Assessment in Graduate Medical Education: Cents and Sensibilities

Cited by 22 publications

References 20 publications

Robotic Minimally Invasive Surgical skill assessment based on automated video-analysis motion studies

Robotic Minimally Invasive Surgical skill assessment based on automated video-analysis motion studies

Review article: Closing the research gap at the interface of learning and clinical practice

Test of Integrated Professional Skills: Objective Structured Clinical Examination/Simulation Hybrid Assessment of Obstetrics-Gynecology Residents' Skill Integration

Contact Info

Product

Resources

About