Daily Encounter Cards—Evaluating the Quality of Documented Assessments

Cheung, Warren J.; Dudek, Nancy; Wood, Timothy; Frank, Jason R.

doi:10.4300/jgme-d-15-00505.1

Cited by 17 publications

(25 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Two previous studies seeking to provide validity evidence for the CCERR found that DECs and ITERs that had been judged by experts to be of 'average' quality received mean CCERR scores of 24.2 and 23.8, respectively. 25,26 The mean CCERR scores for all three groups in our study were lower, suggesting that the quality of DECs completed by supervisors fell below a standard that would be considered to represent 'average' quality. Although this is disappointing, our results are congruent with those of other studies which demonstrate that clinical supervisors consistently fail to complete WBAs to an adequate standard, as represented by similarly low mean CCERR scores (range: [15][16][17][18][19][20][21][22].…”

Section: Quality Of Decs and Resident Performancementioning

confidence: 59%

“…11,12,25 A recent study demonstrated evidence for the validity of CCERR scores when applied to DECs, including high reliability and the ability to discriminate DECs of varying quality. 26 In that study, mean CCERR scores for DECs that had been judged by experts to be of poor, average and high quality were 14.4, 24.2 and 37.3, respectively. 26…”

Section: Outcome Measurementioning

confidence: 82%

“…26 In that study, mean CCERR scores for DECs that had been judged by experts to be of poor, average and high quality were 14.4, 24.2 and 37.3, respectively. 26…”

Section: Outcome Measurementioning

confidence: 82%

See 2 more Smart Citations

Supervisor-trainee continuity and the quality of work-based assessments

et al. 2017

View full text Add to dashboard Cite

Supervisor-trainee continuity did not have an educationally meaningful influence on the quality of assessments documented on DECs. However, resident performance was found to affect assessor behaviours in the On-service group, whereas DEC quality remained poor regardless of performance in the Off-service group. The findings suggest that greater attention should be given to determining ways of improving the quality of assessments reported for off-service residents, as well as for those residents demonstrating appropriate clinical competence progression.

show abstract

Section: Quality Of Decs and Resident Performancementioning

confidence: 59%

Section: Outcome Measurementioning

confidence: 82%

See 1 more Smart Citation

Supervisor-trainee continuity and the quality of work-based assessments

et al. 2017

View full text Add to dashboard Cite

show abstract

“…[9][10][11] Despite their widespread use, evidence suggests that the quality of assessments being documented on DECs is poor. 12,13 A study by Bandiera and Lendrum 11 found that DECs were subject to leniency or range restriction effects wherein supervisors provided "inflated" or overly favorable assessments leading to indiscriminate ratings. Sherbino et al 14 found that DEC items were not well understood by front-line supervisors, resulting in poor reliability and questionable validity of the scores.…”

mentioning

confidence: 99%

The Ottawa Emergency Department Shift Observation Tool (O‐EDShOT): A New Tool for Assessing Resident Competence in the Emergency Department

Cheung

Wood

Gofton

et al. 2019

AEM Education and Training

Self Cite

View full text Add to dashboard Cite

Objectives: The outcome of emergency medicine (EM) training is to produce physicians who can competently run an emergency department (ED) shift. However, there are few tools with supporting validity evidence specifically designed to assess multiple key competencies across an entire shift. The investigators developed and gathered validity evidence for a novel entrustment-based tool to assess a resident's ability to safely run an ED shift. Methods: Through a nominal group technique, local and national stakeholders identified dimensions of performance that are reflective of a competent ED physician and are required to safely manage an ED shift. These were included as items in the Ottawa Emergency Department Shift Observation Tool (O-EDShOT), and each item was scored using an entrustment-based rating scale. The tool was implemented in 2018 at the

show abstract

“…6 Paukert et al 2 noted that they had both resident and faculty evaluators, and noted that as the postgraduate year (PGY) increased, the number of evaluations decreased. POE systems using CECs have been described 4,6-8 with both the benefits seen of the CEC and immediacy of feedback being more useful to students. 7,9…”

Section: Interventionmentioning

confidence: 99%

Point-of-Encounter Assessment: Using Health Belief Model Constructs to Change Grading Behaviors

McLean

Francis

Lacy

et al. 2019

Journal of Medical Education and Curricular Development

View full text Add to dashboard Cite

Objective: Fourth-year medical students need feedback to improve. Even during 1-month rotations, there needs to be a formal mid-clerkship feedback session. Better feedback involves multiple surgical evaluators at multiple levels. Constructs from the Health Belief Model of behavior change were used to assess faculty and resident grading behaviors to create a more usable evaluation system. A point-of-encounter (POE) system was created. The objective of this study was to review the efficacy of a POE clinical evaluation card (CEC) system which was initiated to increase evaluator’s participation in grading and formative feedback prior to mid-clerkship evaluation. Design: The study was a 1-year retrospective cohort study reviewing the CECs for level of evaluator, content, and student compliance. A Likert-type scale survey regarding the usage of the clinical cards was also completed by evaluators. Setting: Texas Tech University Health Sciences Center at El Paso, during 2 fourth-year medical student rotations, Subinternship (Sub-I), and Surgical Intensive Care Unit (SICU). Participants: 34 fourth-year medical students and 20 evaluators. Results: Students turned in a mean of 10 cards, 75% in SICU and 65% in Sub-I turned in all 10 cards. There were significantly greater advanced residents evaluating during Sub-I vs SICU: mean evals by PGY3 were 1.9 vs 0.75 ( p = .01) and mean evals by PGY5 were 1.4 vs 0.1 ( p < .0001). There were significantly more faculty completing evaluations during SICU vs Sub-I: 2.5 faculty evals/student vs 1.4 faculty evals/student ( p = .023). Evaluator ratings were high on a 5-point Likert-type scale, with most responses near the “strongly agree” rating of 4.7 to 4.8. Conclusions: Use of POE CECs met goals of having at least 7 CECs turned in by mid-clerkship and 10 at end-clerkships. Formative evaluations by mid-clerkship went from 0 to 7 evaluations. Evaluator surveys highlighted clarity and efficiency as reasons for using CECs.

show abstract

Daily Encounter Cards—Evaluating the Quality of Documented Assessments

Cited by 17 publications

References 18 publications

Supervisor-trainee continuity and the quality of work-based assessments

Supervisor-trainee continuity and the quality of work-based assessments

The Ottawa Emergency Department Shift Observation Tool (O‐EDShOT): A New Tool for Assessing Resident Competence in the Emergency Department

Point-of-Encounter Assessment: Using Health Belief Model Constructs to Change Grading Behaviors

Contact Info

Product

Resources

About