Quality of dictated feedback associated with SIMPL operative assessments of pediatric surgical trainees

Zendejas, Benjamin; Toprak, Ayca; Harrington, Amanda W.; Lillehei, C. Walton; Modi, Biren P.

doi:10.1016/j.amjsurg.2020.10.014

Cited by 8 publications

(5 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The feasibility and clinical applicability of SIMPL in the operative setting has been previously documented, but use of this tool in other settings has not yet been studied. [17][18][19][20][21] Implementation of the SIMPL application in the ED setting may enhance feedback and assessment.…”

Section: Backg Rou N Dmentioning

confidence: 99%

Implementation of the SIMPL (Society for Improving Medical Professional Learning) performance assessment tool in the emergency department: A pilot study

Haas

Davis

Harvey

et al. 2023

AEM Education and Training

View full text Add to dashboard Cite

Background Feedback and assessment are difficult to provide in the emergency department (ED) setting despite their critical importance for competency‐based education, and traditional end‐of‐shift evaluations (ESEs) alone may be inadequate. The SIMPL (Society for Improving Medical Professional Learning) mobile application has been successfully implemented and studied in the operative setting for surgical training programs as a point‐of‐care tool that incorporates three assessment scales in addition to dictated feedback. SIMPL may represent a viable tool for enhancing workplace‐based feedback and assessment in emergency medicine (EM). Methods We implemented SIMPL at a 4‐year EM residency program during a pilot study from March to June 2021 for observable activities such as medical resuscitations and related procedures. Faculty and residents underwent formal rater training prior to launch and were asked to complete surveys regarding the SIMPL app's content, usability, and future directions at the end of the pilot. Results A total of 36/58 (62%) of faculty completed at least one evaluation, for a total of 190 evaluations and an average of three evaluations per faculty. Faculty initiated 130/190 (68%) and residents initiated 60/190 (32%) evaluations. Ninety‐one percent included dictated feedback. A total of 45/54 (83%) residents received at least one evaluation, with an average of 3.5 evaluations per resident. Residents generally agreed that SIMPL increased the quality of feedback received and that they valued dictated feedback. Residents generally did not value the numerical feedback provided from SIMPL. Relative to the residents, faculty overall responded more positively toward SIMPL. The pilot generated several suggestions to inform the optimization of the next version of SIMPL for EM training programs. Conclusions The SIMPL app, originally developed for use in surgical training programs, can be implemented for use in EM residency programs, has positive support from faculty, and may provide important adjunct information beyond current ESEs.

show abstract

Section: Backg Rou N Dmentioning

confidence: 99%

Implementation of the SIMPL (Society for Improving Medical Professional Learning) performance assessment tool in the emergency department: A pilot study

Haas

Davis

Harvey

et al. 2023

AEM Education and Training

View full text Add to dashboard Cite

show abstract

“…For example, mobile applications geared toward timely and specific feedback exist 56 and may be harnessed for neurosurgical procedures, 57 though recent reports suggest a substantial portion of comments are ineffective in other specialties. 58,59 Fortunately, use of the operating microscope or exoscope in neurosurgery lowers the barrier for personal or mentored film study. Defining an optimal system for truly timely feedback should be a priority among neurosurgery programs.…”

Section: Evidence-poor Domainsmentioning

confidence: 99%

Data-Driven Residency Training: A Scoping Review of Educational Interventions for Neurosurgery Residency Programs

Kelly¹,

Yengo‐Kahn²,

Roth³

et al. 2021

Neurosurg.

View full text Add to dashboard Cite

BACKGROUND Neurosurgery residency programs are tasked with imparting large volumes of both clinical knowledge and technical skill to trainees in limited time. Many investigators have described local practices, which may offer evidence-based interventions in neurosurgical residency education, but this literature has not been systematically reviewed. OBJECTIVE To perform a scoping review of educational practices in neurosurgery, which are supported by quantitative, peer-reviewed research. METHODS A scoping review of the literature was performed. PubMed, Embase, and Web of Science databases were queried for articles describing educational interventions for neurosurgery residents, which included a quantitative assessment of the effect on resident performance. RESULTS From an initial set of 1785 unique articles, 29 studies were ultimately screened and included. Studies were into the following 6 topics: (1) didactics and curricula (n = 13), (2) nontechnical skills (n = 6), (3) wellness and burnout (n = 4), (4) assessment and feedback (n = 2), (5) mentorship and career development (n = 2), and (6) research (n = 2). Individual study results were described. CONCLUSION Several educational interventions in neurosurgical training are supported by quantitative evidence. Methodological shortcomings are prevalent among studies of education, particularly in the selection of meaningful outcome measures. A summary of evidence-based considerations is provided for current and future program directors.

show abstract

“…6,7 High-quality elements of written feedback include information based on direct observation, specific details to assist with recall of events, balanced remarks in providing areas of strength and improvement, and providing suggestions so that residents can change. 2,7,8 Opportunities to provide written feedback vary depending upon the training program. Faculty assessment forms (also known as end of rotation assessments and in-training evaluation reports) are typically delivered to faculty assessors through the use of residency management software.…”

Section: Introductionmentioning

confidence: 99%

A Document Analysis of Nationally Available Faculty Assessment Forms of Resident Performance

French

Pien

2021

Journal of Graduate Medical Education

View full text Add to dashboard Cite

Background Written feedback by faculty of resident performance is valuable when it includes components based on assessment for learning. However, it is not clear how often assessment forms include these components for summative and formative feedback. Objective To analyze prompts used in forms for faculty assessment of resident performance, guided by best practices in survey research methodology, self-regulation theory, and competency-based assessment. Methods A document analysis, which is a qualitative approach used to analyze content and structure of texts, was completed on assessment forms nationally available in MedHub. Due to the number of forms available, only internal medicine and surgery specialties were included. A document summary form was created to analyze the assessments. The summary form guided researchers through the analysis. Results Forty-eight forms were reviewed, each from a unique residency program. All forms provided a textbox for comments, and 54% made this textbox required for assessment completion. Eighty-three percent of assessments placed the open textbox at the end of the form. One-third of forms contained a simple prompt, “Comments,” for the narrative section. Fifteen percent of forms included a box to check if the information on the form had been discussed with the resident. Fifty percent of the assessments were unclear if they were meant to be formative or summative in nature. Conclusions Our document analysis of assessment forms revealed they do not always follow best practices in survey design for narrative sections, nor do they universally address elements deemed important for promotion of self-regulation and competency-based assessment.

show abstract

Quality of dictated feedback associated with SIMPL operative assessments of pediatric surgical trainees

Cited by 8 publications

References 28 publications

Implementation of the SIMPL (Society for Improving Medical Professional Learning) performance assessment tool in the emergency department: A pilot study

Implementation of the SIMPL (Society for Improving Medical Professional Learning) performance assessment tool in the emergency department: A pilot study

Data-Driven Residency Training: A Scoping Review of Educational Interventions for Neurosurgery Residency Programs

A Document Analysis of Nationally Available Faculty Assessment Forms of Resident Performance

Contact Info

Product

Resources

About